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Introduction 

Annex  A,  B,  and  C  contain  three  sets  of  data  tables  organized  in  terms  of  ORO  goals.  Annex  D 
illustrates  how  ORO  goals  taken  from  Annex  A-C  may  be  cross-referenced  to  a  specific  threat 
related  activity  sequence. 

•  Coming  on  watch  goals  -  Annex  A 

These  data  tables  describe  the  goals  of  the  ORO  when  coming  on  watch.  This  is  the  time 
when  the  ORO  must  be  updated  about  information  that  has  arrived  or  events  that  have 
occurred  since  he  or  she  was  last  on  watch.  This  information  must  be  reviewed  and  related 
to  likely  mission  needs  during  the  coming  watch.  The  status  and  schedule  of  ongoing  tasks 
must  be  determined.  The  capability  of  the  ship  and  the  TG  must  be  reviewed.  Critical  or 
complex  issues  or  procedures  may  need  to  be  refreshed.  Members  of  the  incoming  watch 
must  be  briefed  and  focussed  for  the  coming  watch  and  the  ORO’s  command  presence 
established. 

•  Background  (OR  management)  goals  -  Annex  B 

These  data  tables  describe  the  ongoing  goals  of  the  ORO  that  must  be  pursued  during  any 
watch.  These  goals  include  managing  the  OR  team,  dealing  with  the  information  contained 
m  the  text  or  audio  messages  that  come  into  the  OR,  building  and  maintaining  awareness  of 
all  pertinent  information.  Successful  attainment  of  these  goals  prepare  the  ORO  and  the  OR 
to  face  future  threats. 

•  Foreground  (threat  related)  goals  -  Annex  C 

These  data  tables  describe  the  goals  of  the  ORO  during  a  generic  threat  response.  Exactly 
which  goals  are  pursued  will  vary  with  the  type  and  circumstances  of  the  threat.  For 
example,  establishing  communication  with  (and  warning  off)  a  potentially  hostile  contact  may 
not  be  appropriate  once  more  aggressive  Rules  of  Engagement  have  been  established.  These 
goals  may  also  overlap  with  some  background  goals.  For  example,  the  threat  related  goal  of 
maximizing,  optimizing  and  correlating  information  from  different  sensors  concerning  one 
particular  threat  contact  overlaps  with  the  background  task  of  managing  the  OR  team. 
Conversely,  information  garnered  during  the  background  task  of  monitoring  incoming  text 
messages  may  affect  interpretation  of  or  reaction  to  a  potential  threat. 

•  Scenario  event  sequences  -  Annex  D 

In  this  Annex,  activities  for  two  scenario  events  requiring  active  engagement  (i.e.  firing  of 
weapons)  are  tabulated  and  cross  referenced  to  the  data  tables  in  Annex  A,  B,  and  C. 

In  the  comments  section  of  some  data  tables  ad  hoc  observations  about  design  or  research  issues 
have  been  inserted.  Design  and  research  issues  were  not  systematically  reviewed  for  each  ORO 
goal  but  these  comments  were  considered  worth  leaving  in. 


Key:  "  "  .  ' . ~ 

RT:  Response  Time  to  complete  some  task  or  achieve  some  criterion. 

%:  The  level  of  completeness  and  accuracy  achieved  with  respect  to  an  area  of  interest. 

Marie  Celeste  :  RT  to  become  aware  of  ‘situation’  from  scratch  using  only  the  information  accessible 
system  and/or  %  awareness  achieved  after  a  given  time  -  using  only  the  resources  of  the  system. 
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Annex  A: 

COMING  ON  WATCH  GOALS 


Coming  on  Watch  goals: 


Introduction 

Normal  staffing  is  three  qualified  OROs,  one  as  Combat  Officer  plus  two  junior  OROs  working  watch  and  watch  about. 
At  Action  Stations  the  Combat  Officer  and  off  watch  ORO  work  in  support  of  the  ORO  on  watch,  performing  tasks 
under  his/her  direction.  The  Combat  Officer  is  responsible  for  longer  range  planning  i.e.  days  ahead.  OROs  are 
involved  in  earlier  mission  planning  so  will  bring  to  the  watch  a  mental  model  of  the  overall  mission  intent  and  plan,  its 
schedule,  activities  to  date,  background  intelligence  about  goals,  enemy  threats,  etc.  and  the  various  compromises  the 
plan(s)  represent.  Each  ORO  arrives  on  watch  with  a  mental  model  of  what  to  expect  during  the  next  watch  in  terms  of 
the  mission  and  for  the  routine  during  any  watch  at  that  time  of  day  (e.g.  certain  status  reports  or  forecasts  such  as  the 
ATO  arrive  within  certain  time  brackets  each  day.)  Each  ORO  updates  and  refines  his  or  her  expectations  during  the 
process  outlined  below  as  he  or  she  comes  on  watch,  looking  ahead  for  8-12  hours. 

The  ORO  will  first  check  outside  (weather,  sea-state,  other  vessels  around,  coastline,  etc)  before  coming  into  OR  to  take 
over  the  watch.  The  ORO  wants  to  be  able  to  visualize  what  is  outside  the  OR  and  may  go  onto  bridge  to  check  with 
OOW.  (May  also  review  other  sources  such  as  daily  orders,  or  hear  rumours  before  coming  to  OR.) 

Prior  to  getting  a  briefing  from  the  outgoing  ORO,  the  incoming  ORO  walks  around  the  OR  to  check  on  particular 
people  and  each  OR  team,  look  over  state  boards,  review  messages  that  have  arrived  since  last  on  watch  role,  etc.  The 
ORO  will  already  have  a  mental  model  of  capabilities  of  his  or  her  own  OR  team,  including  the  OOW.  Reviewing 
messages  can  be  time-consuming  and  may  take  15-20  minutes.  During  a  2-3  week  exercise  it  was  reported  that  there 
could  be  as  many  as  20,000  text  messages  coming  into  the  OR,  in  addition  to  any  voice  traffic.  A  lot  of  time  is  spent 
extracting  information  from  text  messages  and  entering  this  into  briefing  notes  or  aide  memoirs.  It  is  very  easy  to  miss 
relevant  information  because  of  volume.  The  ORO  needs  to  have  awareness  of  context  to  select  relevant  items.  This 
awareness  of  context  usually  arises  from  being  involved  in  planning.  The  ORO  may  get  several  6-8  page  messages 
while  on  watch.  Messages  go  into  a  series  of  binders  (about  10)  covering  different  aspects  of  operations. 

The  data  tables  in  this  Annex  deal  with  the  following  ORO  goals  when  coming  on  watch.  These  may  overlap  and  are 
only  in  rough  order  of  performance. 

1.  Visualize  external  environment  for  the  start  of  the  watch. 

2.  Update  own  knowledge  for  coming  watch  from  text  msg  information. 

3.  Establish  schedule  of  anticipated  events  for  watch. 

4.  Update  understanding  of  threats. 

5.  Establish  current  activity  levels  and  work  focus  among  OR  team(s) 

6.  Prepare  check-list  to  monitor  progress  on  required  action  items  during  watch 

7.  Review  OR  system  performance  (sensors,  etc)  and  rectify  or  work  around  as  required 

8.  Comprehend  items  in  CO ’s  order  book. 

9.  Comprehend  brief  from  outgoing  ORO 

10.  Optimize  own  CCS  . 

11.  Focus  OR  team  on  issues  for  coming  watch.  Establish  command  presence. 

12.  Review  pertinent  pre-plans,  procedures  and  ROE  for  coming  watch. 
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1  -  ORO  eoalfs):  Visualize  external  environment  for  the  start  of  the  watch. 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Obtain,  picture  and  retain  pertinent  details  of  external  environment  immediately  prior  to  coming  on  watch, 

-  Update  this  picture  as  required 


Awareness 

needs 


Decisions 

required 


Pertinent  features  to  look  for*  Select  pertinent  info. 
Visual  picture  of: 

-weather, 

-  sea  state, 

-  visibility, 

-  cloud  cover, 

-  moon  phase. 

-  visible  ships, 

-  coastline 

other  local  factors. 


Comments: 

ORO  goes  on  deck  or  to  bridge  10  minutes  before  going  on  watch. 
Maybe  difficult  to  simulate  for  test. 

Could  ORO  qet  this  from  within  OR  with  cameras  e.q.  STIR? 


Communication 

sources 


Look  around, 
ask  bridge  team. 


Workload  Measure 

issues  %  =  complete  +  accurate 


SA: 

•  %  env  details 
CM: 

•  Comms  @  environment: 
-# 

-  duration 

-  content  (?) 

DM: 

•  %  categories  covered. 


Method 
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2  -  ORO  fjoalfsl:  UDdate  own  knowledge  for  coming  watch  from  text  msg  information. 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Review  information  in  text  messages  that  have  arrived  since  last  on  watch. 

•  Select  and  retain  pertinent  information  (including  action  required  or  implied)  for  use  during  watch. 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Anvthina  affectinq: 

-  ship  or  TG  capability 

-  sensors,  wpns, 

-  comms,  a/c) 

-  mission  plans 
-schedule  for  watch. 

-  decision  context 
_ROE, 

-  en  threat  profiles, 

-  political  news.) 

•  Divide  bgr  vs  action  info. 

•  Has  action  item  been 
dealt  with? 

•  Who  has  seen  msg? 

•  Who  should  see  msg? 

•  Immediate  vs  delayed 
action  required. 

•  Relevant  during  watch? 

•  Info  put  on  state  boards? 

•  Short  list  items  to  check? 

ORO  msg  file 

Msg  binders 

ORO  turnover  book. 
Highlighted  notes  on  msg 
file. 

Outgoing  ORO 

Takes  10-20  mins  to  go 
through  msg  file. 

ORO  has  priority  for  msg 
file  but  others  coming  on 
watch  also  need  to  see. 

SA: 

•  %  critical  text  msg 
content. 

•  %  retrieval  key  info 
during  watch. 

CM: 

•  Time  to  check  N  msgs. 

•  #  queries  @  msg  text.. 

DM: 

•  %  info  correctly  sorted  fo 
action,  retention  or 
retrieval. 

SA: 

•  Freeze  probe  before 
watch 

•  Embedded  probe  (e.g. 
sitrep)during  watch. 

CM: 

•  SME  or  S#  review 
-real  time  or  video. 

DM: 

•  Freeze  probe  before 
watch. 

•  SME  or  S#  review 

-  real  time  or  video. 

Comments: 

Msgs  follow  a  daily  cycle  i.e.  certain  msgs  expected  during  watch  e.g.  ATO.  Seldom  time  to  get  through  back  volume  of  msgs.  Mission  relevant  info  often  missed. 

Info  that  may  be  of  use  later  rather  than  immediately  significant  may  be  overlooked.  Try  to  remember  ROEs  but  usually  consult  ROE  book  because  so  important. 
Explicit  action  items  in  txt  messages  easier  to  recognize  and  record  than  implicit  action  items  (e.g.  to  forward  info  to  others,  ensure  boards  updated,  etc). 

•  Cognition: 

OROs  have  std  scanning  procedure  for  text  messages.  They  scan  subject  line  and  then  priority  scan  for  info  based  on  knowledge  of  message  format  or  headings 
under  which  expect  information  to  be  organized.  Novice  more  likely  to  take  notes. 

•  Mental  Models: 

ORO  prioritizes  search  and  retention  of  information  using  mental  model  for  mission  task(s)  that  will  occur  during  watch. 

Simultaneous  scanning  for  multiple  types  of  relevant  information  can  be  error  prone. 

•  Other:  . 

Research  issue:  understand  more  about  how  scan  messages  for  mission  relevant  information.  Use  to  improve  the  message  format  design  to  support  ORO  s  needs. 
Help  ORO  to  prioritize  parts  of  msg  to  attend,  retain,  distribute  based  on  mission  needs  for  watch? 
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3  -  ORO  aoalfs):  Establish  schedule  of  anticipated  events  for  watch. 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Identify,  prioritize  and  retain  schedule  related  information  for  the  coming  watch. 

•  Plan  and  deploy  own  and  OR  tm  resources. 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Routine  events 

-  report  deadlines 

-  incoming  reports 

•  Special  events 
-flying  program 

-  conferences 

•  Mission  events 

-  pre-planned  actions 

•  Navigation  route  plan 

•  Prioritize  events. 

•  Plan  deployment 

-  own  efforts 

-  OR  team  efforts 

•  State  board(s) 
e.g.  Fleet  Exercise 
Program  (FLEX) 

•  Daily  orders? 

•  Text  messages. 

•  Outgoing  ORO. 

Use  this  knowledge  to 
manage  own  and  others 
workload  during  watch:  i.e. 
to  predict  likely  workload 
ahead  of  time  and  prepare. 

SA: 

•  %  schedule  of  events 

•  RT  to  criterion 

CM: 

•  Comms  @  schedule 

-  #  queries 

-  duration 

-  content(?) 

DM: 

•  %  scheduling  decisions 

•  %  scheduled  responses 

SA: 

•  Freeze  probe  before 
watch 

•  Embedded  probe  during 
watch 

-  sitrep 

-  missed  events 

CM: 

•  SME  or  S#  review 

-  real  time  or  video. 

DM: 

•  Freeze  probe  before 
watch. 

•  SME  or  S#  review 

-  real  time  or  video. 

Comments: 

Could  be  assessed  in  a  low  fidelity  simulation  environment. 

•  Cognition: 

Number  of  items  to  remember  and  organize?  How  do  OROs  visualize  and  remember  schedule  for  watch.  Is  this  same  for  all  OR  members?  How  to  select  and 
record  anticipated  events  in  a  way  that  facilitates  recall  and/or  monitoring  completion  at  appropriate  time.  Novice  more  likely  to  take  notes. 

•  ORO  problems: 

Tend  to  forget  scheduled  items  under  high  stress.  ORO  needs  to  embed  schedule  related  info  in  other  info  sources,  e.g.  own  msg  binder. 
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4  -  ORO  goal:  Update  understanding  of  threats 

ate  way: 

at  related  information. 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accur 
•  Identify,  comnrehend  and  retain  for  future  use,  any  changes  to  thre 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  All  threat  and  threat 
profiles 

•  Changes  to  above  since 
last  watch 

•  Significance  for  mission 

•  Brief  anyone? 

-(CO,  OR,  TG) 

•  Change  anything? 

-  readiness? 

-  sensor  settings? 
-ROE? 

-  Pre  plans? 

Message  binders 

Threat  board 

Outgoing  ORO 

SA: 

•  %  threats 

•  %  threat  changes  since 
last  watch 

•  %threat  info  used  on 
watch 

CM: 

•  RT  to  criterion 

•  Comms  @  threats 

-  #  comms 

-  duration 

-  content(?) 

DM: 

•  %  threat  implications  for 
watch  ahead 

•  %  threat  data  recalled 
appropriately  during 
watch 

SA: 

•  Freeze  probe  before 
watch 

•  Embedded  probe  during 
watch 

-  sitrep 

CM: 

•  SME  or  S#  review 

-  real  time  or  video. 

DM: 

•  Freeze  probe  before 
watch. 

•  SME  or  S#  review 

-  real  time  or  video. 

Comments: 

ORO  needs  to  embed  threat  related  info  in  other  info  sources,  e.g.  msg  binders.  May  be  threat  info  that  has  not  reached  threat  board. 
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5  -  ORO  goal:  Establish  current  activity  levels  and  work  focus  among  OR  teamts 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Identify  the  current  tasking  of  each  OR  team. 

•  Evaluate  and  re-focus  OR  team  efforts  as  required  to  meet  mission  goals. 


Awareness 

needs 


•  Mission  related  needs  for 
OR  effort  for  watch 
ahead. 

•  Current  activities  in  OR. 

•  OR  resources  available 
now  and  later 


Decisions 

required 


Communication 

sources 


•  Overhear  OR  tms  on  net 

•  Walk  @  OR  and  ask 

•  Look  @  from  ORO  CCS. 

•  ORS,  SWC,  etc. 


Workload 


Measure 

%  =  complete  +  accurate 


SA: 

•  %0R  tm  activities 

•  %  mission  needs 

•  %OR  resource  available 
CM: 

•  RT  to  criterion 

•  Comms  @  tm  activities 

-  #  comms 

-  duration 

-  content(?) 

DM: 

•  %  task  quality  OK 

•  %  refocussing  of  tasks 

-  before  &  during  watch 

•  RT  for  decision  on  watch 


Method 


Comments: 

For  evaluation,  need  scenario  with  range  of  activities  among  OR  tm  that  vary  in  appropriateness  for  upcoming  watch  that  ORO  has  to  identify  and  re-assiqn. 
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COMING  ON  WATCH  GOALS 


6  .  ORO  moal:  PreDare  check  list  to  monitor  Droaress  on  required  action  items  during  watch 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Identify,  and  retain  for  use  during  the  watch  ahead,  information  about  tasks  to  be  completed. 

•  Monitor  task  comDletion  during  watch  without  degrading  other  ORO  tasks  (eg  threat  response). 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  All  potential  action  items 
for  follow  up  on  watch. 

•  Prioritize  items 

•  Create  short  list. 

•  Confirm  status 

-  in  hand  and  OK 

-  in  hand  not  OK 

-  to  be  assigned 

•  Delegate  to? 

•  Establish  deadlines. 

Msg  file. 

ORO  turnover  book. 
Outgoing  ORO. 

OR  tm  members  eg  ORS 
Daily  orders 

Message  cycle. 

Mission  plan 

Used  to  manage  workload 
of  self  and  others. 

SA: 

•  %  action  items 

•  %  status 

CM: 

•  RT  to  criterion 

•  Comms  @  action  items 

-  #  comms 

-  duration 
-content(?) 

DM: 

•  %  Priorities  x  item. 

•  %  follow  up  on  watch 

•  %  delegation  OK 

•  %  schedule  OK 

SA: 

•  Freeze  probe  before 
watch 

•  Embedded  probe  during 
watch 

-  e.g.  query  on  item 
status 

CM: 

•  SME  or  S#  review 

-  real  time  or  video. 

DM: 

•  Freeze  probe  before 
watch. 

•  Embedded  probe  during 
watch 

-  SME  or  S#  review 
real  time  or  video. 

Comments: 

For  evaluation  need  to  consider  a  variety  of  action  items  (at  various  stages  of  progress)  embedded  in  watch  hand-over  material.  Novice  may  take  notes. 

HALIFAXORO:  Annex  A  -  Coming  on  Watch  goals  Humatwystems  Incorporated 


Page  A-8 


I 


Annex  A: 

COMING  ON  WATCH  GOALS 


7  -  ORO  goal:  Review  OR  system  performance  (sensors,  etc)  and  rectify  or  work  around  as  required 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Review,  understand,  and  retain  for  future  reference  information  about  the  capabilities  of  systems  affecting 
(This  includes  repair,  maintenance  schedules,  etc) 


Awareness 

needs 


•  Problems  @  major 
systems  (sensors,  wpns, 
comms,  EW,  helo,  etc).  « 

•  General  nature  of 

problem  &  effect  on  OR 
performance  . 

•  Schedule  for  correction. 

•  Mission  related  priorities 
for  coming  watch 


Decisions 

required 


Match  capabilities  with  < 
likely  needs  over  watch? 
Determine  action 
-  repair,  work  around, 
etc 

Prioritize  action. 


Communication 

sources 


CSE, 

outgoing  ORO, 
pertinent  supervisor 
(EWS,  ORS,  SWC, 
ASWC,  TS,  CIO,  SAC) 
State  boards. 
Observation  of  OR 
equipment 


Workload  Measure 

issues  %  =  complete  +  accurate 


SA: 

•  %  system  deficiencies 

•  %  effect  on  OR 

•  %  schedule  correction 
CM: 

•  RT  to  criterion 

•  Comms  @  deficiencies 

-  #  comms 

-  duration 

-  content 
DM: 

•  %  direction  @  deficiency 
corrections 


OR  operations. 

Method 


SA: 

•  Freeze  probe  before 
watch 

•  Embedded  probe  during 
watch 

-  e.g.  query  on  item 
status 

CM: 

•  SME  or  S#  review 

-  real  time  or  video. 

DM: 

•  Freeze  probe  before 
watch. 

•  Embedded  probe  during 
watch 

-  SME  or  S#  review 
real  time  or  video. 


Comments: 

For  evaluation  need  variety  realistic  system  problems  in  scenario  with  varying  degrees  of  significance  for  upcoming  watch. 
•  Cognition: 

Mostly  retain  in  memory  but  might  take  notes.  Novice  more  likely  to  take  notes. 
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1  s  .  non  anal:  Ooninrehend  items  in  CO’s  order  book. 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Review  understand  and  retain  for  use  during  the  watch,  the  instructions  of  the  CO  for  matters  under  the  dir 

ection  of  the  ORO  . 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Specific  instructions 

•  CO's  intent 

•  Mission  goals 

•  If  -  then  cues 

•  Action  on  this  watch? 

Night  order  book. 

Outgoing  ORO. 

SA: 

•  %  CO  orders 

CM: 

•  RT  to  criterion 

•  Comms  @  CO  orders 

-  #  comms 

-  duration 

-  content(?) 

DM: 

•  %  during  watch 

SA: 

•  Freeze  probe  before 
watch 

CM: 

•  SME  or  S#  review 

-  real  time  or  video. 

DM: 

•  Embedded  probe  during 
watch 

-  SME  or  S#  review 
real  time  or  video. 

Comments: 

Familiar  with  CO’s  style.  Need  to  understand  intent  and  relation  to  mission. 

•  Cognition: 

Alerts  to  prompt  when  related  issue  occurs  during  watch  -  especially  when  under  high  workload. 
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9  -  PRO  goal:  Comprehend  brief  from  outgoing  PRO 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way 

•  Review,  query,  and  retain  for  use  during  the  watch,  information  provided  by  the  outgoing  PRO. 


Decisions 

required 


Communication 
_ sources 

Face  to  face  with  ORO. 
CCS,  GOP,  JMCIS, 
ORO  handover  book 


Workload 
_ issues 

Takes  2-10  minutes 
depending  on  situation. 


Measure 

%  =  complete  +  accurate 
~SA- 

•  %  ORO  hand  over  info 
CM: 

•  Comms  @  hand  over 

-  #  comms 

-  duration  (total  time) 

-  content(?) 

DM: 

•  %  queries  to  outgoing 
ORO 

•  %  appropriate  use  on 
watch 


Method 

'SA 

•  Freeze  probe  after  brief 
CM: 

•  SME  or  S#  review 

-  real  time  or  video. 

DM: 

•  SME  review 

-  real  time  or  video. 

•  Embedded  probe  during 
watch 

-  SME  or  S#  review 

-  real  time  or  video. 
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COMING  ON  WATCH  GOALS 


10  -  ORO  soal:  Optimize  own  CCS  . 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Customize  the  CCS  to  be  used  during  the  watch  ahead  according  to  technical,  personal  and  mission  related 

(Future  CCS  may  offer  options  to  customize  settings  to  individual  needs  or  preferences.  At  present,  these  option 
things  like  brightness,  contrast,  etc.  However,  time  may  come  when  ORO  can  set  up  own  short  cuts,  enable  or  d 
(according  to  own  background  e.g.  SWC  vs  ASWC  or  upcoming  mission  needs),  customize  certain  display  setting 
oceanographic  information  layers)  to  show  different  things  in  different  ways  according  to  different  tactical  situa 
provide  customized  cheat  sheets,  or  communication  networks,  etc,  etc.  The  more  options  offered  by  the  system, 
and  error  prone  will  become  the  process  of  setting  this  up.  CCS  memory  for  personal  settings  that  OROs  coming 
invoke  will  help.)  _ _ _ _ _ _ _ _ _ 

needs. 

s  are  limited  to 
sable  selected  alerts 
rs  (e.g. 

Hons  likely  to  be  met, 
he  more  demanding 
l  on  watch  can 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  + accurate 

Method 

•  Display  options 

•  Mission  needs 

-  Notes  on  pre-plans, 
ROE 

-  info  layers  (e.g. 
Oceanog) 

•  Technical  needs 

-  Procedure  cheat 
sheets 

-  Alarm  settings 

•  First  time  set  up 

-  What  to  choose 

•  Update  for  watch 

-  What  to  choose 

CCS 

Less  time  to  fiddle  with 
display  settings  when  high 
workload.  Desirable  to 
have  pre-set  user  profiles 
for  quick  set  up. 

SA: 

•  %  display  options 

•  %  display  choices  for 
watch 

CM: 

•  Comms  @  CCS  settings 

-  #  comms  (e.g.  #  key 
strokes,  consult  help, 
ask  ORS,  etc) 

-  duration  set  up  (total 
time) 

-comm  content(?) 

DM: 

•  %  setting  choice 

SA: 

•  Freeze  probe  before 
watch 

CM: 

•  SME  or  S#  review 

-  real  time  or  video. 

DM: 

•  Post  hoc  review  of 
settings 

Comments: 

Categories  of  information.  1st  basic  CCS  settings  (brightness,  etc),  2nd  mission  settings  (pre-plans,  ROE,  threat  info,  range  scales,  etc).  3rd  personal  settings 
(procedural  cheat  sheets,  prompts,  alerts,  etc).  These  overlap. 

Useful  to  have  pre-set  options  for  each  user. 

Irrelevant  alarms  maior  source  of  workload.  Need  to  filter  out  for  particular  mission. 
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11  -  ORO  goal:  Focus  OR  team  on  issues  for  coming  watch.  Establish  command  presence. 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Select  information  to  the  OR  team  to  be  taken  into  account  during  the  watch  ahead. 

•  Format  and  provide  this  information  to  the  OR  team. 

•  Ensure  that  petinent  information  provided  has  been  comprehended  by  target  audience. 


Awareness 

Decisions 

Communication 

Workload 

Measure 

Method 

needs 

required 

sources 

issues 

%  =  complete  +  accurate 

•  Watch  handover  info 

•  Sitreps  to  ORO  from 
SWC,  ASWC. 

•  Mission  schedule 

•  Expected  events 

•  Priorities  for  watch 

•  Mood  of  team 

-Need  for  re-assurance 
about  events  ahead. 

•  Feedback  from  tm 

-  comprehension 

-  mood  change 


•  What  to  tell  OR 

-  Technical  factors 

-  Motivational  factors 

•  How  to  tell  OR 
-Mode 

-  Format,  level  of  detail, 

-  Order,  etc 


CCS 

Audio 

Face  to  face 


Less  time  when  high  stress  SA: 

•  %  required  info  (x  ORO) 
CM: 

•  RT  to  criterion 

•  Comms  @  brief 

-  #  queries  x  OR  tm, 

-  #  repetitions  x  ORO 

-  duration  brief  (total 
time) 

-  %  content 

-  comprehension  x  OR 
tm 

-  “tone" 

DM: 

•  %  content 

•  %  manner 


Comments: 

For  evaluation,  “tone"  of  ORO  difficult  to  measure.  Maybe  rating  scale  by  observer  or  target  audience. 

•  Team  management: 

Team(s)  may  get  distracted  from  display/tasks  while  doing  turnover  briefing  and  need  to  double  check  for  items  overlooked. 
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10  -  oro  anal*  Review  pertinent  pre-plans,  procedures  and  ROE  for  coming;  watch. 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Identify  pre-plans,  procedures  and  other  mission  related  information  pertinent  to  the  watch  ahead. 

-  Review  the  identified  material. 

(If  there  is  a  high  likelihood  of,  say,  a  submarine  engagement,  over  the  watch,  ORO  may  want  to  re-fresh  memor 
Which  procedures  will  likely  depend  on  background  of  ORO.  For  example,  former  SWCs  may  be  more  likely  to 
procedures  and  vice  versa.  Novice  OROs  may  want  to  do  this  more  than  experienced  OROs.  System  could facil 
vu>prl?  ORO  rnav  want  to  include  others  in  review.) 

y  of  procedures, 
r eview  ASW 
'tate  such  individual 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Possible  events  for  watc 
and  related: 

-  Procedures 

-  Pre-plans 
-ROE 

•  Likely  events? 

•  Need  to  review? 

•  When  to  review? 

•  How  to  review? 

•  Others  to  include? 

Procedure  check  lists, 
Pre-plan  details 

ROE  details 

GOP,  &/or  CCS 

May  need  rapid  access 
when  implented  in  a  way 
that  is  compatible  with 
priority  task. 

SA: 

•  %  events  +  procedures 
(etc) 

for  watch  ahead 

CM: 

•  Comms  @  review 
-# 

-duration  brief  (total 
time) 

-content(?) 

DM: 

•  %  task  performance  with 
and  without  review. 

SA: 

•  Freeze  probe  before 
watch 

CM: 

•  SME  or  S#  review 

-  real  time  or  video. 

DM: 

•  Post  hoc  SME  review 

•  Task  specific 
performance  measures 
taken  during  watch 

WL: 

Comments: 

For  evaluation  will  need  to  know  whether  effect  of  such  rehearsal  /  review  benefited  performance  of  target  task  during  warn - - - 
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Annex  B: 
BACKGROUND  GOALS 


Background  (OR  management)  goals 


Introduction 

The  work  of  the  ORO  can  be  described  in  terms  of  background  and  foreground  goals.  Background  goals/tasks  are 
primarily  those  associated  with  the  ORO’s  responsibility  to  manage  the  general  preparedness  of  the  OR  to  meet  any 
threat.  Foreground  tasks  are  associated  with  meeting  a  specific  threat  using  the  capabilities  of  the  OR. 

In  general,  the  ORO  does  not  usually  engage  in  the  detailed  response  to  a  particular  threat  but  ensures  that  the  OR 
responds  effectively.  However,  the  exception  to  this  is  when  OR  resources  cannot  handle  simultaneous  threats  without 
the  ORO  taking  over  the  response  to  one  of  the  threats.  Such  a  situation  was  viewed  as  extreme  and  most  likely  to 
involve  multiple  air  threats,  with  the  ORO  acting  as  a  second  SWC.  In  such  cases  the  CO  might  either  take  on  the  OR 
management  duties  relinquished  by  the  ORO  or  the  response  to  one  of  the  threats.  If,  as  seems  likely,  the  ship  is  at 
Action  Stations,  with  the  off- watch  OR  team  and  the  Combat  Officer  present  in  the  OR,  then  the  either  the  off  watch 
ORO  or  the  Combat  Officer  might  assume  overall  responsibility  for  the  OR  team.  This  choice  seems  to  vary  from  ship 
to  ship,  or  rather  from  CO  to  CO.  The  choice  seems  to  hinge  on  the  familiarity  of  the  CO  with  CCS  system  and  the 
style  of  the  CO.  Some  CO’s  prefer  to  leave  their  CCS  and  stand  between  the  ORO  and  the  SWC  so  that  they  can 
observe  both  their  screens  easily  (and  turn  to  observe  the  ASWC  screen  with  little  additional  effort).  This  results  in 
there  being  a  “spare”  CCS  work  station  which  may  be  assigned  to  someone  other  than  the  CO  in  such  circumstances. 

This  Annex  deals  with  the  ORO  background  goals.  These  goals  share  some  of  the  following  characteristics  and 

•  are  always  present  (while  threat  response  requirements  are  intermittent). 

•  take  second  priority  to  threat  response  but  satisfactory  completion  contributes  to  effective  threat  response. 

•  can  be  interrupted  but  a  backlog  of  information  will  then  build  up  that  will  have  to  be  dealt  later  . 

•  require  the  ORO  to  switch  among  them  to  monitor  their  status  and  stay  on  top  of  the  information  stream, 

•  Interact. 

The  data  tables  in  this  Annex  deal  with  the  following  background  goals  of  the  ORO.  They  are  not  mutually  exclusive 
and  are  in  no  particular  order. 

1 .  Monitor  and  manage  the  OR  team  (s ) . 

2.  Manage  information  from  all  sources  coming  to  the  ORO  position  (speech,  text,  other). 

3.  Monitor  and  manage  OR  equipment  capability. 

4.  Build,  monitor  and  maintain  own  awareness  of  the  pertinent  surface  environment. 

5.  Build,  monitor  and  maintain  own  awareness  of  the  pertinent  air  environment. 

6.  Build,  monitor  and  maintain  own  awareness  of  the  pertinent  sub-surface  environment. 

7.  Build,  monitor  and  maintain  own  awareness  of  integrated  picture  (s)  comprising  selected  elements  of  the  surface, 
sub-surface  and  air  environments  and  other  related  information. 

8.  Switch  attention  between  different  tasks  and  re-acquire  awareness  quickly  and  effectively. 

9.  Maintain  awareness  of  ORO’s  authority  to  act. 

1 0.  Check  outgoing  message  traffic. 
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1  -  PRO  goal:  Monitor  and  manage  the  OR  team: 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Ensure  information  flows  among  the  OR  teams. 

•  Determine  and  assign  task  priorities  among  OR  team. 

•  Monitor  task  completion  for  OR  team  in  terms  of  a  critical  path  and  task  quality. 

•  Recognize  or  anticipate  problems 

•  Determine  and  implement  a  solution  with  the  OR  resources  available. 

•  Balance  priorities  for  limited  OR  resources  between  several  parallel  tactical  demands. 

•  Re-allocate  OR  personnel  resources  according  to  tactical  needs. 

Notes; 

The  ORO  has  overall  responsibility  for  the  effectiveness  of  the  on  watch  OR  team  and  monitors  OR  activities  to  that  end.  The 
purpose  is  to  ensure  that  the  OR  team  is  producing  and  distributing  information  and  decisions  as  required  (completeness  and 
accuracy)  and  on  schedule  (timeliness),  in  accordance  with  mission  related  tactical  needs. 

Checks  are  mainly  by  exception  i.e.  the  ORO  assumes  everything  is  satisfactory  unless  there  is  evidence  to  the  contrary.  The  ORO 
has  certain  procedural  expectations  for  the  OR  team  that  certain  things  will  happen  at  certain  points  in  certain  ways  and  may  wait  for 
cues  to  the  contrary.  For  example,  if  an  OR  group  does  not  exhibit  the  activity  level  expected  or  if  information  is  late,  incomplete  or 
ambiguous  (such  as  reports,  responses  not  in  the  form  expected,  a  lag  to  produce  a  symbol  on  the  CCS),  then  the  ORO  will  notice  and 
follow  up.  Follow  up  may  be  direct,  or  via  the  ORS  or  a  warfare  director  (SWC,  ASWC,  SAC). 

More  directly,  the  ORO  assesses  information  or  decision  quality  by  judging  the  suitability  or  completeness  of  a  given  product.  For 
this,  the  ORO  may  use  a  mental  or  actual  checklist  to  ensure  that,  for  example,  the  SWC’s  plan  has  covered  all  the  bases.  Such 
mental  models  are  acquired  over  a  career  of  training  and  exercising  common  procedures,  refined  for  the  team  in  question  and  updated 
during  the  work-up  and  planning  for  the  mission  at  hand  (pre-plans  and  rehearsals).  A  dilemma  is  that  the  ORO  must  pick  up  on  the 
necessary  cues  yet  not  become  too  involved  in  the  detail  (i.e.  interfering  or  becoming  distracted  from  other  ORO  tasks). 


Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Current  mission  goals 

•  OR  tasks  to  be  done 

•  OR  tasks  in  hand 

•  OR  task  schedule 

•  Critical  path. 

•  OR  Resources  available 

•  OR  meeting  tactical 
goals? 

•  OR  tm  members 
-  on  task? 

-on  schedule? 

•  Quality  OK? 

Vis/auditory  view  of  OR. 
SITREPS  from  SWC, 
ASWC,  SAC. 

Feedback  from  ORS 

Data  on  CCS,  boards 

Avoid  too  much  detail 
Balance  OR  management 
with  other  ORO  tasks. 

SA: 

•  %  aware  OR  tasks: 

-  req’d  of  OR  tm 
-on  critical  path 

-  deadlines 
-on  schedule 

-  behind  schedule 
-quality 

•  Comms  to  acquire  SA 
-# 

-  duration 

-  content 

CM: 

•  Comms  @  tasks 
-# 

-  duration 

-  content  (?) 

DM: 

•  For  tasks  on  crit.  Path 

-  %  problems  anticipated 

-  %  problems  recognized 

-  RT  to  problem 

-  %  intervene  OK . 

-  %  deadline(s)  met. 

-  %  resources  idle 

SA: 

•  Freeze  probe 

•  Embedded  probe 
(e.g.  sitrep) 

CM: 

•  SME  review 

-  real  time  or  video. 

-time  line  analysis 

DM: 

•  Freeze  probe 

•  Embedded  probes 

•  SME  review 

-  real  time  or  video. 

continued  on  next  page 
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[Comments: 


Multiple  tasking  a  problem  for  ORO.  Has  to  balance  own  tactical  tasks  with  management  tasks.  (Warfare  directors  doing  same  thing  for  own  tms.)  Need  to 
"visualize"  critical  path  for  tasks  along  way  to  achieving  tactical  goal.  ORO  needs  some  but  not  too  much  insight  into  actions  of  OR  teams. 

•  Potential  problems  .  .  . 

Fail  to  notice  critical  cues  that  something  going  awry.  Critical  path(s)  may  change  during  mission. 

•Relevant  Experience: 

How  long  has  this  team  worked  together?  How  well  do  they  know  /  trust  each  other. 

Important  mental  models  formed  during  mission  planning  and  rehearsal.  Pre-plans  determine  tasks  to  be  done  to  some  extent. 

•Team  issues: 

Information  flow  among  OR  teams.  ORS  and  warfare  directors  key  figures. 

•Critical  Tasks 

Recognize  and  correct  major  tasking  deficiencies  in  OR. 

Analyze  critical  path  to  anticipate  problems  and  meet  deadline(s). 
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2  -  PRO  goal:  Manage  information  from  all  sources  coming  into  PRO  position. 

•  Monitor  all  information  sources  and  select  information  for  own  Situation  Awareness. 

•  Ensure  appropriate  information  flow  within  OR  and  ship,  and  to  TG. 

Criteria: 

For  audio  traffic,  the  ORO  must  be  able,  in  a  timely,  complete  and  accurate  way,  to: 

•  Monitor  all  audio  traffic  and  identify  (anticipate  or  recognize)  traffic  of  interest  for  more  detailed  attention. 

•  Attend  and  interact  with  different  sources  of  auditory  information  at  will. 

•  Identify  source(s)  of  pertinent  audio  traffic. 

•  Understand,  respond,  retain  and/or  redistribute  selected  information  in  a  timely  and  accurate  manner. 

For  text  traffic,  the  ORO  must  be  able,  in  a  timely,  complete  and  accurate  way,  to: 

•  Monitor  all  text  traffic  and  identify  (anticipate  or  recognize)  traffic  of  interest  for  more  detailed  attention. 

•  Review,  understand,  respond,  retain  and/or  redistribute  selected  information  in  a  timely  and  accurate  manner. 

Notes: 

The  ORO  spends  a  great  deal  of  time  managing  information  coming  into  the  OR.  All  sources  of  information  are  potentially  pertinent 
for  the  ORO’s  task  of  building  and  maintaining  awareness  of  factors  affecting  direction  of  the  OR  but  only  two  aspects  considered 
here  are  auditory  traffic  and  text  messages.  This  information  must  be  variously  screened,  filtered,  related  to  the  mission,  retained  for 
future  use,  re-distributed,  and  acted  upon.  Incoming  information  may  be: 

-  Auditory  (comms  nets,  verbal  around  OR,  sound  of  CHAFF  being  fired). 

-  Visual  (text  messages,  CCS  screen,  plot,  JMCS,  state-boards,  body  language  of  team). 

-  Tactile  (ship  movement  for  course  changes)  and  even  olfactory  (smoke). 

Auditory  traffic.  This  lets  the  ORO  know  what  is  going  on  in  the  OR,  on  the  bridge  and  in  the  TG  and  currently  involves  listening 
to  people  talk  around  the  OR,  and  listening  on  headphones  to  various  internal  and  external  circuits.  While  the  headphones  are  on  it  is 
difficult  or  impossible  to  hear  ordinary  speech  in  the  OR.  OROs  manage  such  auditory  information  by  using  the  left  earphone  mainly 
for  the  internal  ship  C2  net  (90%)  while  “bouncing”  to  other  internal  circuits  at  intervals  for  10  -15  seconds  to  snap  shot  other 
activity.  The  ORO  will  also  leave  the  C2  net  when  making  an  announcement  to  the  OR  or  to  the  whole  ship.  The  right  headphone  is 
used  for  external  TG  circuit(s).  Cues  to  bounce  to  monitor  another  net  (ie.  not  the  internal  C2  net)  may  come  from  a  specialist  on  the 
C2  net  (eg  the  SAC),  from  the  ORS,  or  visual  cues  in  the  OR  such  as  sudden  activity  in  one  work  area  or  a  LINK  symbol  appearing 
on  screen.  (Note:  there  is  about  30  second  delay  from  verbal  report  by  operator  to  LINK  symbol  appearing  on  screen.)  Other  reasons 
for  “ bouncing ”  are  to  request,  or  to  provide  information,  or  to  give  direction.  Selecting  communication  circuits  (which  one  to 
monitor)  and  extracting  and  retaining  the  pertinent  information  currently  forms  a  large  part  of  the  ORO’s  workload.  It  is  perceived  as 
difficult  to  manage  and  error  prone. 

Text  traffic.  Managing  text  traffic  during  each  watch  is  an  important,  time  consuming  and  ongoing  responsibility  of  the  ORO. 
Interviewees  reported  that  a  2-3  week  exercise  may  bring  as  many  as  20,000  text  messages  into  the  OR,  in  addition  to  voice  traffic. 
(More  text  traffic  can  be  expected  during  an  actual  operation  than  an  exercise.)  Based  on  exercise  experience,  estimates  were  that 
text  messages  may  come  to  the  ORO  at  a  rate  of  200-300/day  i.e.  about  8-12/hour.  Message  length  may  be  a  few  lines  or  several 
pages.  May  get  several  6-8  page  messages  while  on  watch.  Text  messages  will  often  update  or  cause  changes  to  plans  already  made. 
A  lot  of  time  is  spent  extracting  information  from  text  messages  and  making  briefing  notes  or  aide  memoirs.  For  example,  takes 
ORO  an  hour  to  go  through  the  (daily)  ATO  (Air  Traffic  Order  —  300  pages/day  during  Gulf  War)  with  another  hour  to  reformat  the 
relevant  material  into  a  table.  Interviewees  claimed  it  is  easy  to  miss  relevant  information  because  of  volume.  ORO  needs  awareness 
of  both  message  format  and  mission  context  to  select  relevant  information.  Awareness  of  context  usually  arises  from  being  involved 
in  planning  and  grows  over  the  course  of  the  mission.  Messages  go  into  a  set  of  binders  (about  10)  for  different  aspects  of  operations. 
With  the  ORS  and  signaller  acting  as  filters,  the  ORO  must  read  each  message,  note  (mentally  or  otherwise)  the  details  he  needs  and 
ensure  key  information  reaches  the  OR  team  in  a  timely  manner.  Priorities  for  messages  are  determined  by  several  factors.  Partly  by 
a  classification  assigned  by  the  sender  (FLASH  are  dealt  with  immediately),  partly  by  ORO  workload  priorities,  and  partly  by 
pertinence  to  the  tactical  task(s)  in  hand  e.g.  threat  response.  Length  may  also  be  relevant  since  brief  messages  can  be  scanned  and 
dealt  with  quickly.  During  threat  responses,  a  back  log  will  build  up.  Use  of  JMCS,  JMARTS  still  evolving  e.g.  as  e-mail,  TG 
intranet,  source  of  pictures,  or  graphics.  Presently  photocopies  given  to  each  recipient  who  extracts  own  info  and  puts  msg  into 
binder.  When  coming  on  watch  ORO  must  review  messages  that  have  arrived  since  previous  watch. 

continued  on  next  page 
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Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•Info  flow  around  OR 
•Txt  msg 

-  structure 

-  content 

-  priority 
•Audio  msg 

-  structure 

-  content 

-  priority. 

•Mission  priorities 

-  information  needs 
•Tactical  priorities 

-  information  need 

•  msg  content  relevance? 

-  immediate? 

-  potential? 

•Action  arising? 

-  immediate? 

-schedule? 

-  highlight?  store? 
•Distribute? 

-who? 

-when? 

•Auditory: 

-  Headset(s) 

-  Face  to  face 

-  Clues  from  OR  activity, 
CCS. 

•Txt  msgs 

-  read  and  note 
-Graphics  =  JMCS 

-  ORS  notes  to  ORO 

•Audio: 

90%  time  on  internal. 

15  users  /net 

Major  source  of  workload. 
•Text: 

High  volume  of  txt  msgas. 
Some  very  long  egATO. 
Most  of  normal  watch  is 
spent  reading  msgs. 

Major  source  of  workload. 

For  all  audio  and  txt  msqs 
that  arrived  in  OR 

SA: 

•%  msg  presence 
•%  content  details 
•%  comprehension 
(i.e.  signiffor  mission) 

•%  retention  key  details 
(test  at  intervals). 

CM: 

•Over  total  scenario) 

-  Time  spent  on  msg(s) 

#  queries  @  msg(s) 

#  transcription  errors 

DM: 

•%  re-distribution 
•%  re-action. 

(For  both  audio  and  txt ) 

SA: 

•Freeze  probe 
•Embedded  probe 
(e.g.TG  query) 

CM: 

•SME  review 

-  real  time  or  video, 
-timeline  analysis 
-transcription  errors 

DM: 

•Freeze  probe 
•Embedded  probes 
•SME  review 

-  real  time  or  video. 

Comments: 

•Cognition:  Difficult  to  tran 
of  current  communicatior 
•Memory:  claim  can  reme 
attention  with  other  tasks 
•Visual  info:  spread  ail  ovs 
Audio  info:  fragmented, 
•Potential  problems  Txt 
Transposition  errors  corr 
Important  info  gets  lost  c 
•Relevant  Experience:  Kn 
•Team  issues  Rely  on  OF 
between  ORS,  ORO,  sig 
Need  map  based  framev 
•General:  Presentation  o 
•Measurement  Issues:  R 
(e.g.  scans  CCS).  Seed 
Will  need  SME  input  toe 
the  stack  of  messages  ( 
based  trainer  does  not  s 
•Research:  attention  to  m 

slate  text  based  spatial  info  into  graphic  or  visualized  spatial  picture.  Key  info  needs  to  be  highlighted  in  text  msgs.  Need  further  analysis 
categories  and  patterns. 

Tiber  general  content  of  msgs  but  need  notes  for  details.  Heavy  reliance  on  memory  for  msg  content.  Txt  msg  review  competes  for 
.  Familiarity  with  msg  format  over  long  years  helps  scanning  for  key  information. 

;r  OR  -  must  look  and/or  move  around  to  get  it. 

ransient,  miss  items  if  not  on  net.  Must  transpose  to  other  medium  to  help  retention  -  e.g.  post  it  notes,  state  boards,  CCS. 

nsg  backlog  difficult  to  overcome.  May  contain  info  critical  for  current  situation. 

imon  when  putting  txt  based  spatial  info  into  system  for  display  on  CCS  e.g.  latitude,  course,  speed.  Etc. 

r  overlooked. 

owledge  of  msg  format(s)  aid  scanning  text  info. 

*S  to  point  out  key  info  arriving  but  ORS  has  other  tasks  too.  Effectiveness  in  this  task  depends  heavily  on  common  understanding 
nailers  for  text  msgs  and  among  other  team  members  to  give  ORO  heads  up  to  listen  on  audio, 
vork  on  which  incoming  plans,  overlays,  or  related  info  can  be  selectively  imposed, 
information  can  vary  with  technology  (for  example,  current  text  msgs  might  converted  to  e-mail) 

scord  representative  set  of  incoming  traffic  (audio  and  txt)  during  a  scenario  or  exercise.  Feed  this  to  ORO  while  performing  secondary  task 
scenario  with  x-section  of  errors,  priorities,  msg  lengths. 

ietermine  what  are  “correct"  responses.  Test  for  ability  to  recall  and/or  use  msg  details  after  various  intervals.  Potential  metrics:  clearing 
f  read  per  unit  time  or  #  lines  per  unit  time).  Identifying  relevant/important  messages.  Routing  information  to  the  appropriate  people.  Land 
imulate  text  message  aspect  well  (too  brief  &  too  few).  Need  to  know  more  about  existing  msg  distribution  system, 
ultiple  audio-sources  may  be  better  with  3D  audio? 
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3  -  ORO  goal(s):  Monitor  and  manage  OR  eauiDment  capability  status 

etc)  and  tactical  significance  of  any  shortfall. 

alfunction,  or  enemy  action.  Major  changes 
■ning  the  capability  to  be  fully  functional  and 
>le  to  acquire  and  maintain  awareness  of  or 
capability.  This  means  knowing  what 
t  capability  will  return  and  in  what  manner 
ranges  in  status  and  the  tactical  significance 
ility.  Categories  of  interest  include 
;  ship  maneuver  (engines,  steering,  current 
to  remember  key  areas  of  deficiency,  and 

Criteria:  An  ORO  must  be  able  to.  in  a  timelv.  complete  and  accurate  way 

•  Form  /  maintain  a  mental  picture  of  capabilities  status  (priorities,  schedule  for  repair, 

•  Select  more  detail  about  a  particular  deficiency  as  required. 

•  Determine  and  communicate  priorities  for  repair  or  work  around  solution(s). 

Notes: 

Equipment  status  frequently  changes.  This  may  be  for  reasons  of  routine  maintenance,  m 
that  can  occur  as  a  result  of  enemy  action  will  coincide  with  the  greatest  urgency  for  retui 
the  greatest  conflict  with  other  demands  on  the  ORO.  Consequently,  the  ORO  must  be  al 
at  least  have  ready  access  to  information  about  pertinent  equipment  and  sensor  status  and 
works  on  the  ship  (and  in  the  TG)  and  what  does  not  (in  whole  or  in  part),  and  when  a  los 
(incremental,  all  or  none,  etc).  The  ORO  must  made  aware  promptly  and  accurately  of  cf 
of  such  changes.  The  ORO  must  then  be  able  to  prioritize  efforts  to  regain  any  lost  capab 
weapons  (functionality,  ammunition,  readiness);  sensors  (functionality,  readiness,  tuning) 
evolutions  such  as  helo  operations),  personnel  (skill,  fatigue,  casualties).  The  ORO  needs 
to  get  pertinent  details  quickly  and  easily,  when  required. 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•Current  status  of  major 
eqpt  items  (wpns, 
sensors,  comms,  helo, 
etc) 

•Schedule  for  repair 
•Tactical  implications 
•Changes  in  status. 

•Status  acceptable? 

•Meet  mission  eqpt  needs 
•Priority  for  repair? 
•Tactical  significance  of 
shortfall? 

State  boards 

CSE,  ORS, 

SAC,  SWC,  ASWC 

CCS  data  quality 

May  be  very  high 

SA: 

•%  capability  status 

CM: 

•RT  status  change 
•Comms  @  status 
-# 

-  duration 

-  content  (?) 

DM: 

•%  signif  status  change 
•%  response 

-  repair  priority 

-  notify  others,  etc 

-  work  around  chosen 

SA: 

•Freeze  probe 
•Embedded  probe 
(e.g.  CO’s  query) 

CM: 

•RT  =  Embedded  probe 
•SME  review 

-  real  time  or  video. 

-  time  line  analysis 

DM: 

•Freeze  probe 
•Embedded  probes 
•SME  review 

-  real  time  or  video. 

Comments:  - - - - - - - 

•Evaluation 

What  is  relationship  of  this  to  damage  control? 

•Cognition: 

Equipment  malfunction  or  damage  tends  to  be  unpredictable  -  requires  creative  solutions  at  very  short  notice  often  under  stress. 

•Relevant  Experience: 

Career  path  affects  relative  comprehension  of  different  technical  issues.  For  example,  former  SWCs  less  likely  to  understand  ASW  deficiencies. 
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4  -  PRO  goal(s):  Build  and  maintain  awareness  of  surface  environment - 
f surface  traffic  contacts,  lanes/limits,  coastline,  tides,  sea  state,  navigation,  etc) 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Acquire  and  maintain  awareness  of  significant  surface  issues  to  the  desired  level  of  detail. 

•  Integrate  related  data  elements  into  a  meaningful  view  as  required. 

•  Detect  pertinent  changes  in  key  elements  of  surface  picture  promptly  and  accurately. 

•  Project  surface  picture  into  the  near  future  to  support  navigational  and  tactical  decision  making. 

•  Retrieve  and  comprehend  stored  details  on  specific  issues  in  a  timely  and  appropriate  manner. 

•  Identify  hostile  contacts  beyond  own  weapon  range  and  preparation  time. 

•  Re-acquire  awareness  (detect  changes  since  last  view  and  comprehend  tactical  significance)  of  the  surface  situation  after  switching 

from  another  task,  view  or  level  of  detail. 

Notes? 

The  ORO  has  to  detect  and  comprehend  the  various  surface  issues  that  may  affect  the  ship  and/or  the  TG  in  the  conduct  of  the  mission 
Elements  include  all  other  vessels  within  range  of  own  or  TG  sensors  (other  TG  ships,  helos,  MPA,  etc).  This  includes  own  force, 
enemy  and  civil  contacts.  Other  factors  include  territorial  limits  and  political  implications,  coastline  and  related  navigational  hazards, 
tactical  limits  (minefields,  weapon  ranges  -  own  and  enemy).  The  ORO  must  be  able  to  build,  maintain  and  use  this  awareness  with 
the  least  effort  and  timely  comprehension  of  the  issues  which  arise  for  the  mission,  for  impending  tactical  decisions  and  for 
management  of  the  OR  team. 


Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•All  surface  contacts  within 
relevant  range. 

•Current  contact  status 
(fr,  en,  neutral,  civil) 
•Zones  (channels,  traffic 
patterns,  territory  limits) 
•Hazards 

(wrecks,  shallows,  etc) 
•Threat  profiles 

-  relative  wpn  ranges. 
•Changes  in  contact  or 

track  status. 

•Time  lines  for  approach 
•Pertinent  int 

-  threat  profile 

-  likely  en  locations 
-etc 

•Relative  information 

-  routes 
-CPA,  etc 

•Contact  valid? 

•All  contacts  classified? 
•Threat  analyzed? 

-inside  wpn  range? 
-worst  case? 

-  priority  to  investigate 
•SWC  dealing  OK? 

CCS,  GOP,  JMCS 

Threat  board,  notes 

SWC,  ORS 

TS  and  team 

TG 

New  contacts  usually 
embedded  in  a  contact 
data  stream  that  may  be  as 
many  as  20-30  at  any  give 
moment,  with  new  contacts 
appearing  every  few 
minutes  -  or  - 
Picture  slow  to  build  hence 
=  vigilance  problem.  May 
get  distracted  by  other 
tasks. 

SA: 

•%  current  surface  info, 
-contacts,  limits,  etc 
•%  contact  details 

-  approach,  threat,  etc. 
CM: 

•RT  contact  change(s) 

(or  #  changes  responded 
by  ORO  within  N 
seconds) 

•RT  to  query  @  contact 
(e.g.  from  TG) 

•Comms  @  surf  picture 
(over  scenario  or  per 
contact). 

-# 

-  duration 

-  content  (?) 

-  #  actions  or  time  to 
access  contact  info 
required. 

DM: 

•RT  +  %  tactical 
significance 

•  RT  +%  tactical  response 
•RT  +  %  match  threat 
profile 

SA: 

•Freeze  probe 
•Marie  Celeste  sitrep 
•Embedded  probe 
(e.g.  sitrep  request) 

CM: 

•RT  =  Embedded  probe 
•SME  review 

-  real  time  or  video. 

-  time  line  analysis 

DM: 

•Freeze  probe 
•Embedded  probes 
•SME  review 
-realtime  or  video. 

continued  on  next  page 
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Comments: 

•Evaluation: 

Measure  ability  of  S#  to  walk  into  OR,  sit  down  at  CCS,  and  start  reporting  picture  to  other  (e.g.  CO)  e.g.  time  to  reach  desired  knowledge  state,  or  knowledge  state 
achieved  within  given  time.  Data  to  be  in  system  database  and  accessible. 

Also  compare  with  and  without  access  to  system  to  help  generate  answer  to  probe  questions  e.g.  answer  probe  without  access,  vs  probe  with  access  to  any  OR 
data  source. 

•Cognition 

May  require  skill  at  building  mental  picture  and  to  switch  between  local  and  global  perspectives. 

•Potential  problems 

Currently  hard  to  integrate  separate  elements  required  for  a  classification. 

New  ship  to  ship  missiles  have  longer  ranges  therefore  time  frame  shortening. 

Relevant  info  spread  around  OR  and  can  go  to  ground. 

Surface  picture  takes  2nd  priority  to  air  picture. 

Surface  picture  neglected  if  SWC  busy  with  air  picture,  in  which  case  ORO  may  act  as  back  up.  This  affects  ORO  oversight  of  other  threat  areas  or  global  picture. 
•Relevant  Experience: 

Takes  experience  to  be  able  to  bring  to  together  elements  into  visualized  picture. 

•Team  issues 

SWC  does  80%  work,  ORO  rest. 

Smart  tool  to  assist  contact  classification  e.g.  what  info  needed,  what  info  available 
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BACKGROUND  GOALS 


5  -  PRO  goal(s):  Build  and  maintain  awareness  of  air  environment^ 


j  “  y/JLVVy  • - — — — — - - 

fair  traffic  contacts,  air  traffic  lanes/limits/schedules,  flight  program,  coastal  terrain,  etc) 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 


Acquire  and  maintain  awareness  of  significant  air  issues  to  the  desired  level  of  detail. 

Integrate  related  data  elements  into  a  meaningful  view  as  required. 

Detect  pertinent  changes  in  key  elements  of  air  picture  promptly  and  accurately. 

Project  air  picture  into  the  near  future  to  support  navigational  and  tactical  decision  making. 

Retrieve  and  comprehend  stored  details  on  specific  issues  in  a  timely  and  appropriate  manner. 

Identify  hostile  contacts  beyond  own  weapon  range  and  preparation  time. 

Re-acquire  awareness  (detect  changes  since  last  view  and  comprehend  tactical  significance)  of  the  air  situation  after  switching 
from  another  task,  view  or  level  of  detail. 


Notes: 


The  ORO  must  be  able  to  detect,  comprehend  and/or  recall  the  various  air  issues  that  may  affect  the  ship  and/or  the  TG.  Elements 
include  all  air  contacts  within  range  of  own  or  TG  sensors.  This  includes  own  force,  enemy  and  civil  contacts,  territorial  limits  and 
political  implications,  coastline  terrain,  weapons  ranges  -  own  and  enemy,  and  others.  The  ORO  must  be  able  to  build,  maintain  and 
use  this  awareness  with  the  least  effort  and  timely  comprehension  of  the  issues  which  arise  foi  the  mission,  for  impending  tactical 
decisions  and  for  management  of  the  OR  team.  Monitoring  information  currently  in  ATO  a  major  concern  —  during  Gulf  War,  ATOs 
could  run  to  several  hundred  pages.  Other  air  traffic  activity  includes  scheduled  airline  routes  and  local  flights  (e.g.  oil  platforms  and 
general  aviation)  for  which  flight  plans  may  or  may  not  have  been  submitted  and  available. 


Awareness 

needs 


All  contacts  within  sensor 
or  weapon  range  of  ship 
orTG. 

ATO,  TG  program,  airline 
scheduies. 

■Differentiate  fr,  en, 
neutral,  civil,  unknown. 
■Zones  (air  lanes,  traffic 
patterns  eg  oil  heios) 
•Changes  in  contact  status] 
or  tracks,  altitude,  etc. 
•Threat  profiles 

-  relative  wpn  ranges 

-  Pertinent  int. 

•Pertinent  int 

-  threat  profile 

-  likely  en  locations 
-etc 

Relative  information 

-  approach  rate 

-  routes 
CPA,  etc 


Decisions 

required 


•Contact  valid? 

•All  contacts  classified? 
•Threat  analyzed? 

-  inside  wpn  range? 
-worst  case? 

-  priority  to  investigate 
.SWC  dealing  OK? 


CCS,  GOP,  JMCS 
Threat  board,  notes 
SAC 

SWC,  TS  and  team 
ORS. 

TG 


Communication 

sources 


New  contacts  usually 
embedded  in  a  contact 
data  stream  that  may  be  as] 
many  as  20-30  at  any  give 
moment,  with  new  contacts] 
appearing  every  few 
minutes. 


Workload 

issues 


Measure 

%  =  complete  +  accurate 


ISA: 

%  current  air  data 
%  contact  timelines 
|CM: 

RT  contact  change(s) 

(or  #  changes  responded 
by  ORO  within  N 
seconds) 

RT  to  query  @  contact 
(e.g.  from  TG) 

Comms  @  air  picture 
(over  scenario  or  per 
contact). 

-# 

-  duration 

-  content  (?) 

-#  actions  or  time  to 
access  contact  info 
required. 

DM: 

|»RT  +%  tactical 
significance 

•RT  +  %  tactical  response 
RT  +  %  match  threat 
profile _ 


Method 


SA: 

Freeze  probe 
Marie  Celeste  sitrep 
Embedded  probe 
(e.g.  sitrep  request) 
|CM: 

•RT  =  Embedded  probe 
SME  review 

-  real  time  or  video. 

-  time  line  analysis 
DM: 

Freeze  probe 
Embedded  probes 
•SME  review 
real  time  or  video. 


[Comments: 


•Method: 

Marie  Celeste  -  see  surface  awareness  entry  above. 

Range  of  interest  depends  on  sensor  wpn  technologies  of  the  day. 

,  Sub,  surf  and  air  related  since  missile  may  come  from  either  source. 

•Cognition: 

Accuracy  of  spatial  details  in  mental  models  and  ability  to  communicate  these  effectively  (quickly  and  accurately)  important  for  all  warfare  areas. 
If  complex  manipulation  of  CCS  required  to  modify  picture  or  access  information,  distracts  attention  of  user  away  from  primary  task. 
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Annex  B: 
BACKGROUND  GOALS 


5 


6  -  ORO  soaks):  Build  and  maintain  awareness  of  sub-surface  environment 


sub-surface  contacts  (mines,  submarines,  torpedos.  obstacles),  underwater  terrain,  currents,  thermal  conditions  that  affect  underwater 


sensors,  etc) 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Acquire  and  maintain  awareness  of  significant  sub-surface  issues  to  the  desired  level  of  detail. 

•  Integrate  related  data  elements  into  a  meaningful  view  as  required. 

•  Detect  pertinent  changes  in  key  elements  of  sub-surface  picture  promptly  and  accurately. 

•  Project  sub-surface  picture  into  the  near  future  to  support  navigational  and  tactical  decision  making. 

•  Retrieve  and  comprehend  stored  details  on  specific  issues  in  a  timely  and  appropriate  manner. 

•  Identify  hostile  contacts  beyond  own  weapon  range  and  preparation  time. 

•  Manage  water  space. 

•  Re-acquire  awareness  (detect  changes  since  last  view  and  comprehend  tactical  significance)  of  the  sub-surface  situation  after 

switching  from  another  task,  view  or  level  of  detail. 

Notes: 

The  ORO  must  be  able  to  detect,  comprehend  and/or  recall  the  various  sub-surface  issues  that  may  affect  the  ship  and/or  the  TG. . 
Elements  include  all  sub-surface  contacts  within  range  of  own  or  TG.  This  includes  mines  and  minefield  areas,  submarines,  torpedos, 
underwater  terrain,  territorial  limits  and  political  implications,  underwater  navigational  hazards,  underwater  and  surface  noise  such  as 
engine, ,  weapon  ranges  -  own  and  enemy,  and  others.  The  ORO  must  be  able  to  do  this  with  the  least  effort  and  rapid 
comprehension  of  the  issues  which  arise  for  the  mission,  for  impending  tactical  decisions  and  for  management  of  the  OR  team. 


Awareness 

_ needs _ 

•All  potential  contacts 
•Underwater  topology 

-  obstacles 
-thermal  conditions. 

•No  go  zones  (eg  joint 

action  areas,  sub-action 
areas)  and  duration. 

•  Hazards  (wrecks, 
shallows,  etc) 

•Changes  in  data 
•Threat  profiles 

-  relative  wpn  ranges 

-  Pertinent  int 


Decisions 

_ required 

•ASWC  dealing  OK? 
•Contact  valid? 

•All  contacts  classified? 
•Threat  analysis 

-  inside  wpn  range? 

-  worst  case 

-  priority  investigation? 
•Location  wrt  zone(s)? 


Communication 

sources 

CCS,  GOP,  JMCS. 
Threat  board,  notes 
ASWC. 

ORtm 


Workload  Measure 

issues  %  =  complete  +  accurate 

Must  actively  plot  sub-  SA: 

surface  changes  into  CCS  .%  current  sub  surf  data 


and  GOP. 


May  be  a  lot  of  undeiwater  q^. 
data  to  comprehend  in  --r 
littoral  area  because  of  *, 

surface  traffic  noise.  ;  . 


•%  contact  timelines 


RT  contact  change(s) 


Method 

SA- 

•Freeze  probe 
•Marie  Celeste  sitrep 
•Embedded  probe 
(e.g.  sitrep  request) 


(or  #  changes  responded  CM: 

to  by  ORO  within  N  .  RT  =  Embedded  probe 


Comments: 

•Potential  problems 

Biggest  source  of  error  is  failure  to  update  water  space  management  boxes 
•Relevant  Experience: 

Sub-surface  issues  difficult  to  comprehend  and  visualize  for  ORO  without  sub-surface  background. 


seconds) 

•RT  to  query  @  contact 
(e.g.  from  TG) 

•Comms  @  sub  picture 
(over  scenario  or  per 
contact). 

-# 

-  duration 

-  content  (?) 

-  #  CCS  keystrokes  to 
access  contact  info. 

DM: 

•RT  +  %  tactical 
significance 

•RT  +  %  tactical  response 
•RT  +  %  match  threat 
profile 


•SME  review 

-  real  time  or  video. 

-  time  line  analysis 
DM: 

•Freeze  probe 
•Embedded  probes 
•SME  review 

-  real  time  or  video. 
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Annex  B: 

BACKGROUND  GOALS 


7  -  PRO  goal(s):  Build  and  maintain  awareness  of  integrated  picture(s)  ~ 

(sub-surface,  surface,  air,  resource  availability,  threat  profiles,  mission  time  lines,  etc). 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Acquire  and  maintain  integrated  global  awareness  to  the  desired  level  of  detail. 

•  Integrate  related  data  elements  into  a  meaningftil  view  as  required. 

•  Detect  pertinent  changes  in  key  elements  of  integrated  picture  promptly  and  accurately. 

•  Project  integrated  elements  into  the  near  future  to  support  navigational  and  tactical  decision  making. 

•  Retrieve  and  comprehend  stored  details  on  specific  issues  in  a  timely  and  appropriate  manner. 

•  Identify  and  track  hostile  contacts  from  beyond  own  weapon  range  and  preparation  time. 

•  Switch  in  terms  of  area  of  focus  and  level  of  detail. 

•  Re-acquire  awareness  (detect  changes  since  last  view  and  comprehend  tactical  significance)  of  the  integrated  situation  after 

switching  from  another  task,  view  or  level  of  detail  (zoom  in  or  out). 

Notes:  .  K  .  M 

At  a  high  level  the  ORO  tries  to  maintain  as  current  as  possible  an  integrated  view  of  all  pertinent  mental  models  or  pictures 
(warfare  areas, ’capability,  related  threat  profiles,  ROE).  OROs  may  have  to  iterate  between  several  “views”  to  do  this.  ORO  wants 
to  become  aware  of  changes  and  relationships  in  time  to  respond  to  any  emerging  threat.  This  is  done  by  forming  and  then  reviewing 
the  integrated  high  level  picture  for  concerns  using  whatever  technology  is  available  either  pro-actively  (the  ORO  goes  looking)  or 
passively  (the  system  alerts  the  ORO  to  pay  attention).  Passive  alerts  include  being  made  aware  by  other  members  of  the  OR  team. 
The  “picture”  may  be  organized  graphically,  spatially  or  in  some  other  way.  For  instance,  the  graphic  element  may  be  to  imagine 
actual  coastline  or  to  have  it  represented  by  the  technology  of  the  day  (eg  lines  on  a  CCS  screen).  Issues  affecting  the  ship  and  the 
TG  need  to  be  projected  in  time  out  to  the  end  of  the  watch.  OROs  want  to  select  the  various  issues  to  be  kept  under  review. _ 


Awareness 
_ needs _ 

Features  (above)  of  all 
warfare  areas  but  at 
limited  level  of  detail. 
Relative  importance  of 
different  items  of 
information  (based  on 
imminence,  risk,  etc) 
Important  rel'ships  among 
warfare  areas. 

Other  related  areas  such 
as: 

-  ship  capability 
-TG  capability 
-ROE 

Emerging  situation(s) 


Decisions  Communication 

_ required _ sources 

►When  and  how  to  switch  OR  tm  (voice),  TG  tm 
focus  in  terms  of  area  or  (voice) 
detail?  CCS,  GOP,  JMCS 

►Important  rel'ships  among  State  boards 
warfare  areas? 

•What  to  attend? 


Workload  Measure  Method 

issues  % =  complete  +  accurate 

SA  SA 

•%  ofherw’fare  area(s).  -Freeze  probe 

%  integration  issues  .Marie  Celeste  sitrep 

•%  other  timelines  .Embedded  probe 

CM:  (e.g.  sitrep  request) 

•RT  other  change(s)  CM: 

( or#  changes  responded  ,rj  =  Embedded  probe 


to  by  ORO  within  N 
seconds) 

•  RT  to  query  @  other 
(e.g.  from  TG) 

•Comms  @  other 
(over  scenario). 

-# 

-  duration 

-  content  (?) 

-  #  CCS  keystrokes  to 
access  other  info. 

DM: 

•RT  +  %  tactical 
significance  other 
•RT  +  %  tactical  response 
other 

•RT  +  %  match  threat 
profile  other 
WL: 

•%  other  areas  as  WL  in 
main  area  increases. 
•Subjective  WL  ratings _ 


•SME  review 

-  real  time  or  video. 

-  time  line  analysis 
DM: 

•Freeze  probe 
•Embedded  probes 
•SME  review 

-  real  time  or  video. 
WL 

•Freeze  probes 
•Embedded  probes 
•Rating  scale 


continued  on  next  page 
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Annex  B: 
BACKGROUND  GOALS 


Comments:  ' 

•Evaluation: 

Scenario  will  need  plenty  of  “other"  data  on  which  to  probe. 

Run  scenario  with  focus  in  one  threat  area  and  probe  for  knowledge  in  others. 

Cognition  Information  overload  and  data  clutter  is  a  problem 
•Questions: 

Does  the  ORO  want  access  to  separate  mental  pictures  for  air,  surface,  and  subsurface  or  just  one  integrated  picture?  Or  all  three?)  In  a  multi-threat  scenario  (e.g. 
surface  and  air),  can  the  ORO  relate  different  pictures  to  judge  priorities?  How  do  people  link  the  different  pictures  in  their  minds? 

Does  the  ORO  need  one  picture  or  several?  If  the  ORO  has  some  integrated  mental  model  or  picture,  what  are  the  elements?  Maintaining  multiple  mental 
representation  has  greater  consequences  for  mental  resources  than  maintaining  a  single,  more  complex  model.  This  is  a  research  issue:  what  are  implications  of 
maintaining  multiple  mental  models,  A  single  model  has  advantages  for  seeing  relationships  between  elements  but  management  of  level  of  detail  required  will  be  a 
key.  Can  ORO  pay  attention  to  all  aspects  of  situation  at  once  (geo,  political,  civilian/neutral,  task  force).  OROs  from  different  backgrounds  may  want  to  configure 
threat  pictures  from  different  warfare  areas  differently. 
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Annex  B: 

BACKGROUND  GOALS 


S  -  PRO  goal(s):  Switch  attention  between  different  “pictures”  and  acquire  awareness  with  minimum  delay. 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Delimit  global  or  local  picture(s)  as  required  (select  elements  to  include-  warfare  area,  equipment  status,  OR  tasks,  etc). 

•  Switch  attention  at  will  quickly  and  easily  between  global  picture  to  local  picture,  or  local  to  local. 

•  Zoom  in  or  out  of  any  given  picture  quickly  and  easily  without  distraction. 

•  Cross  reference  between  pictures  quickly  and  accurately  without  distraction. 

•  Acquire  awareness  (detect  changes  since  last  view  and  comprehend  tactical  significance)  of  newly  attended  picture. 
Notes: 


The  ORO  needs  to  switch  focus  between  pictures  and  to  “’’zoom”  in  and  out  at  will  as  events  evolve.  Examples  include:  to  oversee 
the  threat  of  the  moment  from  the  perspective  of  the  warfare  director  responsible.  Zoom  out  in  terms  of  time  or  range  and  bearing  to  a 
new  contact  or  look  for  new  contacts,  or  as  a  missile  progresses  along  its  path  towards  its  target.  To  check  on  some  capability  status 
issue  or  the  progress  of  some  task  within  the  OR.  The  ORO  needs  control  over  the  way  in  which  information  is  layered,  combined  andl 


Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurati 

•Where  to  focus? 

All  mission  related  sources 

SA: 

-  Factors  affecting  and 

•What  to  select  for 

of  information 

•%  new  picture  after  std 

linkages  among  three 

combination? 

See  serials  above 

inspection  time  (e.g.  5 

warfare  areas. 

•How  much  detail? 

seconds). 

-  Data + events  @  broad 

•When  to  switch? 

•%  +  RT  detect  data 

time/space  horizon. 

•Local  oicture: 

•What’s  new? 

changes  in  one  picture 
while  attending  other. 

one  warfare  area  or  one 

•What’s  significant? 

•%  relational  data. 

part  of  one  area. 

CM: 

(see  serials  above) 

•Time  to: 

•Picture  of  others  in  OR 

-  detect  need  to  switch 

-  consistency 

from  onset  new  data  in 

-  focus. 

other  picture 

-  details 

-  switch  to  new  picture 

•Other  related  info 

-  report  on  new  picture 

-  resources 

-fine  tune  new  picture 

-  OR  taskings 

•#  comms  to: 

-etc. 

-  clarify  new  picture. 

DM: 

•%  switches. 

•#/type  switch  errors 

Method 


ISA: 

Freeze  probe 
Embedded  probe 
(e.g.  sitrep  request) 
|CM: 

•RT  =  Embedded  probe 
•SME  review 

-  real  time  or  video. 

-  time  line  analysis 
DM: 

•Embedded  probes 
•SME  review 

-  real  time  or  video. 
|WL 

Freeze  probes 
Embedded  probes 
Rating  scale 


Comments: 


Need  better  CCS  navigation  skills  trg. 

•Evaluation  Method: 

Direct  ORO  to  go  to  particular  focus  with  particular  levels  of  detail  -  eg  when  asked  by  CO. 

•Cognition 

Problem  to  re-establish  “picture"  as  switch  attention.  May  take  several  minutes.  Need  to  know  when  to  switch. 

Miss  changed  info  (updated  or  new)  when  switch  to  new  picture. 

Confusion  of  elements  across  pictures. 

Interference  between  pictures. 

•Potential  problems 
Regain  focus  after  switch. 

Ranges  often  mistaken. 

Sometimes  forget  some  info  layers  suppressed. 

•Relevant  Experience:  _  _  . 

Keeping  straight  in  one's  mind  an  evolving  situation  and  its  related  non-spatial  elements  (resources,  tasking,  etc)  is  a  skill  that  takes  on-task  experience. 

•Team  issues  ,  .  .,  ,  .  , 

The  need  to  interact  with  other  team  members  (ship  or  TG)  about  same  problem  and  to  have  the  same  elements  in  mind.  Confusing  if  other  person  has  ditterent 

image  (focus,  elements,  range,  etc). 

•Design/Research  issues 

This  goal  might  be  achieved  by  changing  pictures  on  a  single  screen  or  switching  gaze  between  pictures  presented  simultaneously  on  adjacent  displays. 
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9  -  PRO  goal(s):  Maintain  awareness  of  ORO’s  authority  to  act. 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Acquire  and  maintain  awareness  of  limits  to  own  authority  to  act  independently  of  any  superior  authority. 

•  Know  how  limits  to  authority  are  changed  by  circumstances. 


Notes: 

The  ORO  needs  to  know  at  any  time  his  freedom  of  action  without  reference  to  superior  authority.  This  understanding  will  be  based 
on  information  about  ROEs,  the  CO’s  instructions  (long  standing  and  for  the  immediate  events  in  the  period  ahead),  and  pre-plans 
prepared  during  mission  planning  to  deal  with  anticipated  events.  There  will  also  be  standardized  procedures  determined  during 
training  and  work  ups  such  as  ZIPPO  procedures.  Some  aspects  of  authority  are  affected  by  circumstances.  For  example,  ROEs  can 
change  automatically  from  defensive  to  aggressive  as  a  result  of  a  hostile  act  on  the  part  of  a  contact  -  such  as  opening  fire  on  one’s 
own  ship  or  another  ship  in  the  TG.  Freedom  of  action  for  the  ORO  will  also  be  constrained  to  some  extent  by  procedures.  (If  long 
standing  procedures  are  changed  at  short  notice  there  is  potential  for  confusion  among  other  team  members,  especially  under  stress  or 
time  pressure.) 


Awareness  Decisions 

_ needs _ required 

CO’s  current  orders  about  »Free  to  act? 
freedom  to  act.  .ROE  changed? 

•Current  ROE 
•Factors  changing  ROE. 


Communication 
_ sources 

CO  night  book, 
state  board,  notes. 

CO,  TG 
ROE  manual 
Text  messages 
Mission  briefing 


Workload  Measure 

issues  %  =  complete  +  accurate 


Knowledge  usually  needed  SA: 
when  under  pressure  of  .%  freedom  to  act  at  given 
high  workfoad  and  combat  points  in  scenario 
stress.  Qfy/t; 

•Comms  @  freedom  (over 
scenario  or  per  contact). 
-# 

-  duration 

-  content  (?) 

DM: 

•%  action 


Method 


Comments: 
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10  -  ORO  goalfsl:  Check  outgoing  message  traffic. 

ate  way: 
ry  and  schedule 

o  on  time,  are  correctly  addressed,  and  have  correct  content.  The 
and  intervene  if  not  satisfactory.  The  ORO  requires  outline  status 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accur 

•  Call  up  back  log  of  outgoing  message  traffic  by  message  catego 

•  Check  selected  messages  for  completeness  and  accuracy. 

Notes: 

The  ORO  is  responsible  for  ensuring  that  messages  sent  from  OR  g 
ORO  must  monitor  activity  of  OR  comms  team  responsible  for  this 
of  message  traffic  so  can  detect  and  diagnose  problems. 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•Backlog  in  msg  traffic  by 
msg  priority. 

•Urgent  msgs  gone? 

•No  back  log? 

•Follow  up  needed? 

ORS,  CIO(?) 

Message  binder? 

SA: 

•%  status  out  msgs 

CM: 

•Comms  @  out  msgs 
-# 

-  duration 

-  content  (?) 

DM: 

•%  out  problems 

SA: 

•Freeze  probe 
•Marie  Celeste 
•Embedded  probe 
(e.g.  sitrep  @  msgs) 

CM: 

•RT  =  Embedded  probe 
•SME  review 
-  real  time  or  video, 
-timeline  analysis 

DM: 

•SME  review 
-realtime  or  video. 

Comments: 

Mostly  a  problem  for  flagship. 

•Evaluation  Method: 

Generate  a  set  of  outgoing  msg  traffic. 

Have  signallers  omit  to  send  x-section  of  msgs  during  scenario  and  check  if  ORO  realizes. 
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Annex  C: 
FOREGROUND  GOALS 


Foreground  (threat  related)  goals 


Introduction 

This  Annex  deals  with  generic  ORO  responsibilities  for  a  threat  requiring  active  engagement.  In  outline  this  sequence 
involves  the  following:  the  ship  (or  TG)  detects  a  threat  amongst  the  ongoing  stream  of  contacts  coming  into  the  OR, 
prepares  and  implements  a  planned  response  which  may  involve  communicating  with  the  contact,  comes  to  Action 
Stations,  fires  the  ships  weapons,  sustains  and  deal  with  damage  to  the  ship,  assesses  damage  to  the  target,  prepares  and 
submits  reports  and  stands  down.  There  may  or  may  not  be  a  requirement  to  deal  with  a  second  threat  that  emerges 
during  implementation  of  the  planned  response  to  the  original  threat.  The  data  tables  that  follow  describe  goals  and 
awareness  needs  at  a  level  that  is  common  to  all  three  warfare  areas:  sub-surface,  surface  and  air.  Note  that  aspects  of 
one  or  more  “Background  Tasks”  (described  in  Annex  B)  may  be  required  during  the  threat  response.  These  have  not 
been  repeated  but  do  overlap  with  some  of  the  tasks  in  this  Annex.  Likely  examples  include  building  and  maintaining 
global  awareness,  managing  the  OR  team,  coordinating  with  other  TG  members,  and  switching  attention  between 
different  tasks  and  re-acquiring  awareness  (i.e.  coming  up  to  speed  quickly  after  each  switch).  In  particular,  there  will 
be  a  need  to  regain  global  awareness  after  focussing  on  a  particular  threat. 

The  following  assumptions  are  made. 

•  The  ORO  is  working  in  a  OR  team  that  has  worked  up  together,  and  shares  common  mental  models  about 
procedures,  pre-plans,  expected  threats  (etc)  derived  from  active  participation  in  the  planning,  training  and 
rehearsals  for  this  mission. 

•  The  ORO  has  been  through  the  coming  on  watch  tasks  and  is  presently  attending  to  a  range  of  background  tasks. 

•  The  OR  team  is  busy  with  a  stream  of  air,  surface  and  subsurface  contacts  that  are  being  resolved  and  classified. 
Air  and  surface  contacts  could  each  be  as  frequent  as  10-  20/hour  in  a  busy  coastal  setting.  Subsurface  work  will 
relate  more  to  oceanographic  issues,  surface  contacts,  and  other  noise,  rather  than  a  high  volume  of  actual 
contacts. 

•  The  ORO  only  pays  attention  to  a  particular  contact  if  the  OR  team  in  question  draws  his  attention  to  that  contact, 
based  on  certain  contact  features  (established  for  the  mission  in  question).  In  the  absence  of  such  an 
announcement,  the  ORO  gets  on  with  background  tasks,  occasionally  refreshing  his  global  picture  by  consulting 
his  CCS,  or  contacting  (audio  or  visit)  individual  warfare  departments  or  the  GOP.  The  threat  response  sequence 
starts  once  the  attention  of  the  ORO  is  drawn  to  a  particular  contact  by  the  SWC  or  ASWC. 

For  any  contact,  there  will  be  a  number  of  features  about  which  information  is  needed  (course,  speed,  bearing,  distance, 
image  strength,  correlation  with  data  from  other  sensors,  distance  from  known  enemy  bases,  match  with  known  threat 
profiles,  electronic  emissions,  related  radio  chatter,  etc).  At  the  initial  contact,  there  will  be  only  a  limited  set  of  data 
available  to  address  such  factors.  Once  there  is  sufficient  evidence  to  suggest/confirm  a  possible  threat,  the  ORO  will  be 
formally  notified  by  the  SWC  or  ASWC  (though  informal  awareness  that  something  is  happening  may  precede  this).  The 
ORO  will  assume  (but  want  to  confirm)  that  the  OR  team  is  working  to  activate  all  potential  data  sources,  to  refine  the 
data  from  these  sources  as  it  becomes  available  (e.g.  by  fine  tuning  the  sensors),  and  look  for  complementary  information 
among  potentially  related  data  sources.  (For  example,  interpretation  of  subsurface  data  on  engine  noises  to  relate  to  a 
surface  contact  to  narrow  down  possibilities  about  vessel  type.) 

Once  the  contact  is  identified  as  having  sufficient  threat  potential  to  claim  more  of  the  ORO’s  attention,  ongoing 
background  tasks  conducted  by  the  ORO  may  continue  but  with  a  different  distribution  of  the  ORO’s  effort.  For 
example,  tasks  such  as  reviewing  incoming  text  messages  may  be  dropped  and  a  backlog  allowed  to  build,  except  for 
threat  related  messages.  This  re-distribution  of  effort  will  depend  on  supplementary  resources  available  within  the  OR  at 
the  time.  Depending  on  the  state  of  readiness  (e.g.  whether  the  ship  is  at  Action  Stations),  time  of  day,  and  other 
factors,  other  personnel  may  be  in  the  OR  and  available  to  take  over  appropriate  tasks.  For  example,  the  CO  and/or  the 
Combat  Officer  may  be  on  ad  hoc  visits  or  all  off  watch  OR  personnel  may  be  there  if  the  ship  is  at  Action  Stations. 
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Annex  C: 

FOREGROUND  GOALS 


The  following  ORO  goals  are  considered  in  this  Annex,  shown  in  rough  order  of  implementation.  Not  all  items  may  be 
required  for  any  given  threat  response.  For  example,  depending  on  the  status  of  ROE,  the  clarity  of  the  classification  of 
the  threat,  and  the  risk  to  collateral  damage  to  white  shipping,  there  may  not  be  any  need  to  attempt  to  communicate 
with  the  contact.  In  keeping  with  the  need  for  the  ORO  to  multi-task,  many  of  the  goals  below  will  be  pursued  in 
parallel,  with  frequent  switches  between  several  tasks.  The  sequence  may  also  vary. 

A:  Classify  contact  and  establish  threat  levels 

1.  Integrate  new  contact  information  with  own  global  picture 

2.  Ensure  OR  prepares  quick  first  response,  broad  plan  for  worst  case. 

3.  Optimize  work  among  OR  team(s) 

4.  Optimize  information  flow  within  and  outside  OR. 

5.  Maximize  number  of  information  sources  concerning  contact 

6.  Optimize  data  quality  for  each  information  source 

7.  Establish  communication  with  contact  to  resolve  identity  and  intentions 

8.  Conduct  warnings  in  accordance  with  TG  tasking. 

9.  Ensure  contact  classified  correctly 

B:  Prepare  plan 

10.  Ensure  OR  detects  if  en.  missile  (etc)  fired  and  establish  firm  track. 

11.  Ensure  detailed  viable  tactical  plan  prepared. 

12.  Check  warfare  directors  (WD)  plan. 

13.  Make  CO  aware  of  all  relevant  components  of  the  plan.  Get  feedback  from  CO. 

14.  Ensure  all  OR  team  aware  of  plan.  Emphasize  key  points. 

C:  Implement  plan 

15.  Supervise  execution  of  plan  by  OR  team. 

16.  Ensure  standard  procedures  completed  correctly  (Greyhound,  ZIPPO,  Action  stations,  ASMD,  etc). 

17.  Stay  on  top  of  global  picture  during  threat  response. 

18.  Back  up  WD  director  as  required. 

19.  Maneuver  ship  as  required. 

20.  Check  for  other  threats 

21.  Respond  to  second  threat 

D:  Assess  engagement  outcome 

22.  Confirm  enemy  target  destroyed 

23.  Assess  damage  to  own  ship  from  missile  (etc)  hit. 

E:  Return  to  surveillance  role 

24.  Regain  awareness  of  the  global  picture  following  period  of  concentration  at  threat  level. 

25.  Re-focus  team.  Establish  command  presence. 

26.  Stand  down  weapons 

27.  Stand  down  ship 

28.  Complete  and  send  reports. 
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Annoy  P ■ 

FOREGROUND  GOALS 


1-ORO  Goal(s):  Integrate  new  contact  information  with  own  global  Dicture 

tasks  in  OR 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  integrate  contact  information  into  global  awareness  quickly  and  accurately. 

•  establish  potential  threat  to  ship,  TG,  and  mission. 

•  determine  priorities  for  OR  attention  (including  own)  by  comparison  with  on-going 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Global  picture: 

(See  bgr  tasks) 

•  Contact  details 

-  co,  spd,  bearing,  etc 

-  range,  CPA,  etc 

-  rel’n  to  ship,  TG,  etc 

•  Threat  profiles 

•  Mission 

-  goals 
-ROE 

•  Current  OR  tm  tasks 

-  priority 

-  schedule 

•  Initial  assessment 
-worst  case? 

-  any  action? 

•  Flag  to  attend  later? 

-CCS 

-Threat  board 

-  SWC,  ASWC,  SAC 
-TG 

-ORS 

-  GOP,  JMCIS. 

SA: 

•  RT  to  contact 

•  %  known  contact  inf 

•  %  fit  contact  &  global 
picture. 

CM: 

•  comms  @  BP: 

-# 

-  duration 

-  content  (?) 

•  comms  @  contact: 

-# 

-  duration 

-  content 

DM: 

•  %  ^assessment 

•  %  action  required 

•  %  in  establishing  prioritie 

•  RT  1st  assessment 

SA: 

•  Embedded  probe(s) 

•  Freeze  probe(s) 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Embedded  probes 

•  SME  review 

-  real  time  or  video. 

Comments:  ~ - - 

•Cognition: 

Significant  memory  load  for  all  that  has  to  be  kept  in  mind:-  threat  profiles,-  eqpt ,  resource  status,  -  recent  events 

Problem  to  visualize  various  weapon  ranges  (enemy  and  friendly). 

Attention  -  OROs  develop  a  scanning  strategy  to  take  in  relevant  inforamtion. 

Multi-tasking  a  challenge  for  ORO. 

•  Potential  Problems: 

Currently  too  much  verbal  description  of  spatial  info  among  OR  team  members.  This  risks  transposition  errors  and  is  slow  to  visualize. 

•  Experience: 

Experience  required  to  establish  appropriate  level  of  attention/  workload  to  this  task. 

Inexperienced  ORO  could  take  too  much  time  at  this  stage. 

•  Evaluation 

May  need  to  pre-load  with  ongoing  secondary  task  to  mimic  background  workload  from  which  switch  attention  to  contact. 

•  Research  /  design  issues 

How  do  OROs  form  and  manage  multiple  global  picture  and  local  picture(s). 

ORO  wants  better  frame  of  reference  for  SA  of  location  -  land,  islands,  seabed  features,  wrecks,  magnetic  anomalies,  etc. 

Want  to  be  able  to  select  layers  of  information 
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Annex  C: 

FOREGROUND  GOALS 


2-ORO  eoalfsV.  Ensure  OR  prepares  auick  first  response,  broad  plan  for  worst  case 

tiore  detailed  tactical  plan,  only  selected 

Y- 

Criteria:  An  ORO  must  be  able  to,  Ln  a  timely,  complete  and  accurate  way: 

•  Anticipate  the  worst  case  implications  as  quickly  as  possible. 

•  Determine  at  least  first  steps  for  broad  plan  to  cover  worst  case. 

•  Implement  immediate  emergency  action  required,  if  any. 

•  Identify  deadline  for  detailed  plan  preparation  and  implementation 

•  Initiate  detailed  planning. 

Note: 

While  the  generic  sources  of  information  for  the  broad  plan  are  the  same  as  those  for  the  n 
information  is  typically  used  to  arrive  at  a  speedy  solution  that  looks  at  the  worst  case  onl; 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  All  contact  info 

-  CPA,  schedule,  etc 

•  All  threat  info 

-  capability 
-wpn  ranges 

-  apparent  intent 

-  possible  tactics 

•  Mission  information 
-TG/CO  intent/tasking 
-ROE. 

-  COA  &  pre-plans 

-  procedures 

•  Own  resources 

-  availability 

-  capability 
(e.g.  wpn  ranges) 

-  time  available 

•  WD's  estimate 

•  Worst  case? 

•  COA? 

-fits  pre-plan? 

-  alternatives? 

•  Immediate  action? 

-  maneuver? 

-  change  readiness? 

•  Ship  /  OR  ready? 

-  need  to  do? 

•  Free  to  act? 

-  obtain  tasking 

-  ask  for  new  ROE? 

•  Deadline(s)? 

-  time  appreciation? 

•  WD's  estimate  OK? 

•  Need  to  brief? 

-CO,  OR,  OOW,  TG. 

CCS,  State  boards, 

GOP,  JMCS 

OR  w'fare  tm  leader(s) 

CO,  TG. 

Need  to  do  this  as  rapidly 
as  possible. 

Very  little  time. 

Start  OR/ship  moving  in 
right  direction  and  refine  as 
moving. 

SA: 

•  %  contact  info 

•  %  threat  info 

•  %  mission,  TG  intent 

•  %  COA 

•  %  pre-plans 

•  %  fr  capability 

CM: 

•  comms  @  plan 
-# 

-  duration 

-  content  (?) 

DM: 

•  RT  to  plan 

•  %  plan  correct 

SA: 

•  Freeze  probes 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 

-  real  time  or  video. 

•  Planning  duration  from 
ORO  alerted  to  contact  t 
start  of  brief  to  CO. 

Comments: 

Immediate  action  required  to  get  ship  moving  in  right  direction  and  other  broad  preparation,  just  in  case,  then  refine  details. 
OR  team  uses  TAC  talk  that  all  know  to  save  time  and  achieve  precision. 

May  be  5-8  pre-plans  to  consider.  Will  use  cheat  sheets  to  cover  off  pre-plan  details. 

ORO  might  not  start  some  tasks  until  later,  depends  on  time  apprecetiation. 

•Experience: 

ORO  will  have  understanding  with  CO  of  need  to  consult. 

Familiarization  with  pre-plans  and  procedures  during  mission  planning  and  rehearsal. 

Experience  teaches  what  information  to  gather  to  satisfy  all  critical  Situation  Awareness  needs. 

•Challenges: 

Calculate  CPA 
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1  3-ORO  eoalfsl:  Optimize  work  among  OR  teamfst 

rate  way: 

s  (task,  priority,  schedule,  workload,  spare  capacity,  etc) 
e  (i.e.  to  under-resourced  and  overloaded  areas) 
m  of  watch). 

)f  monitoring  and  managing  the  OR  team.  Ongoing  throughout 

Criteria:  An  ORO  m 

•  acquire  and  mai 

•  re-assign  OR  re 

•  anticipate  resou 
Note: 

This  can  be  seen  as  a 
threat  response. 

ust  be  able  to,  in  a  timely,  complete  and  accu 
ntain  awareness  of  tasking  of  all  OR  resource 
sources  appropriately  as  circumstances  chang 
rce  allocation  needs  in  the  near  future  (duratic 

particular  case  of  the  ORO  background  task 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  New  contact  related 
tasks 

•  Ongoing  tasks  in  OR 

-  Relative  priorities 

•  Present  workload  in  OR 

•  OR  spare  capacity 

•  OR  resource  needs 
-now 

-  later 

•  Critical  path  for  all  tasks 

•  ORtm(s)  OK? 

-  on  right  tasks? 

-  re-task? 

-on  schedule? 

•  ORO  drop  bgr  tasks? 

-  focus  on  contact? 
-continue  existing  task? 

•  Brief  OR  yet? 

SWC,  ASWC,  SAC, 

ORS, 

Audio, 

visual  check  in  OR. 

CCS 

Depends  on: 

-  contact  volume 

-  other  OR  tm  tasks. 

-  own  bgr  tasks 

-  OR  readiness 

-  who  is  in  OR. 

May  not  pay  much  attentio 
when  initial  contact  info 
comes  in  -  waits  for  more 
details. 

SA: 

%  tasks  required 
%  critical  path. 

%  work  load  all  OR  tms. 

%  spare  OR  capacity 
%  resource  needs 

CM: 

comms  @  OR  task  status: 
-# 

-  duration 

-  content  (?) 

DM: 

%  schedule  solutions 
%  resource  solutions 
%  task  drop/add. 

%  task  assignment 

RT  for  problem  on  critical 
oath 

RT  to  missed  task  deadline 

SA: 

•  Freeze  probes 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Embedded  probes 

•  SME  review 

-  real  time  or  video. 

comments:  - - — - - - 

May  want  to  review  TADMUS  OR  management  module. 

•Cognition: 

Rely  on  memory  for  timelines  and  tasks. 

•Challenges: 

Other,  higher  priority  needs  may  interfere 

•Team  Management 

Team  already  busy  with  contact  stream  in  busy  coastal  waters.  Tend  to  manage  OR  by  absence  of  action  or  report:  prompt  if  no  report  when  expected. 

ORO  will  be  familiar  with  strengths  and  weaknesses  of  different  members  of  OR  team. 

Common  mind  set  among  OR  team  based  on  mission  planning  and  rehearsals.  All  familiar  with  pre-plans  and  procedures 

Critical  path  analysis  involves: 

-  goal(s) 

-  task(s)  required  in  series  /  parallel 

-  resources  /  task 

-task  schedule/deadlines 

-  risk  points 

-  task  progress 
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Annex  C: 
FOREGROUND  GOALS 


5-ORO  goalfsfr  Maximize  number  of  information  sources  concerning  contact 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  check  information  source  options. 

•  check  access  to  information  from  available  sources. 

•  re-direct  OR  team(s)  to  ensure  all  available  information  sources  being  used. 


Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

•  Info  'matrix'  for  contact 
-co,  spd,  bearing,  etc 

-  range,  CPA,  etc 

-  rel'n  to  ship,  TG,  etc 
«  Data  status  of  matrix 
-obtained  vs  required. 

•  ORtm  OK? 

-working  on  matrix? 

•  ORO  action  required? 

-  prompt  tm? 

-  allocate  resources? 

CCS,  GOP,  JMCS 

OR  team  leaders 

ORS,  OOW 

Other  TG  assets 

Cheat  sheets 

Status  boards 

SA: 

•  %  info  required  for  matrix 

•  %  matrix  status 

•  %  OR  matrix  activity 

CM: 

•  comms  @  info  matrix: 

-# 

-  duration 

-  content  (?) 

DM: 

•  RT  matrix  problem 

•  %  matrix  problems 

•  %  matrix  reactions 

SA: 

•  Freeze  probes 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 

-  real  time  or  video. 

Comments:  ~  ~  ~ 

Concept  of  information  'matrix"  shorthand  for  ‘all  the  information  needed  and  sources  that  can  contribute". 

This  and  next  step  should  take  no  longer  than  60  sec. 

OROs  presently  use  cursor  to  locate  items  of  interest  (e.g.  helo  and  contact)  and  to  help  visualize  spatial  relationship  with  ship 
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FOREGROUND  GOALS 


6  -  ORO  soaks'!:  Optimize  data  aualitv  for  each  information  source 

Criterion:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  monitor  acceptability  of  data  quality  for  each  data  source. 

•  identify  options  to  improve  data  and  information  quality. 

•  direct  and  monitor  improvement  data  quality. 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +•  accurate 

Method 

•  Data  quality 

-  completeness 

-  accuracy 

•  Data  sources 
-technical  factors 
(e.g.  sensor  capability) 

-  environmental  factors 
(e.g.  oceanography) 

•  OR  team  actions 

•  Data  OK? 

-  complete? 

-  accurate? 

•  Sensors  OK? 
-technically? 

-  environmental? 

•  ORtm  OK? 

-  working  as  req’d? 

•  ORO  action  required? 

-  prompt  tm? 

-  maneuver? 

-  more  resources? 

CCS,  GOP,  JMCS 

OR  tm  leaders 

ORS,  OOW 

Other  TG  assets 

SA: 

•  %  data  quality  x  source 

•  %  OR  fine  tune  options 
CM: 

•  comms  @  fine  tuning: 

-# 

-  duration 

-  content  (?) 

DM: 

•  RT  fine  tune  problem 

•  %  fine  tune  problems 

SA: 

•  Freeze  probes 

•  Embedded  probes 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 
-real  time  or  video. 

Comments: 

•Team  mgt: 

May  share  responsibility  for  global  vs  threat  awareness  with  other  ships  on  TG.  That  is,  one  ship  will  concentrate  on  contact,  others  will  “watch  its  back"  in  other 
warfare  areas. 
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7  -  PRO  goal(s)  Establish  communication  with  contact  to  resolve  identity  and  intentions 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  identify  communication  options 

•  determine  communication  plan 


Awareness 

Decisions 

Communication 

Workload 

Measure 

Method 

needs 

required 

sources 

issues 

%  =  complete  +  accurate 

Pre-plan  options 
Warnings  required 
ROE 

Contact  options 
Contact  attempts 
Contact  response 
-verbal 
-  behaviour 


•  How  to  communicate? 

-  means? 

-who? 

-  timeline? 

-warning  text? 

-  relay? 

•  Comms  tech  OK? 


Contact 

CCS, 

State  boards 
OR  tm  members 
ORS,  00W 
Other  TG  assets 
ORO  notes 


Comments: 


Standard  warning  text  to  be  read. 

TG  command  ship  may  undertake  or  delegate. 


SA: 

•  %  comms  options 

•  %contact  response(s) 
CM: 

•  comms  @  warnings 
-# 

-  duration 

-  %  content 
DM: 

•  %contacts 

•  RT  comms  problems 

•  %  decisions 

-  comms  choice 

-  warning  schedule 
-warning  evaluation 


SA: 

•  Freeze  probes 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 

-  real  time  or  video. 
DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 

-  real  time  or  video. 
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Annex  C: 

FOREGROUND  GOALS 


8  -  PRO  goal  (s')  Conduct  warnings  in  accordance  with  TG  taskin 


Criterion:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  prepare  warning  plan. 

•  assign  responsibilities  for  plan  implementation  within  OR. 

•  monitor  implementation  of  warning  plan  (assess  communication  quality,  contact  responses,  etc)  and  re-direct  as  necessary. 


Awareness 

needs 


•  Warnings  required 

-  pre-plans 

•  Contact  response(s) 

-  verbal 

-  behaviour 

•  ROE 

•  Global  picture 


Decisions 

required 


Warning  plan  OK? 

OR  resources  OK? 

OR  team  ready? 
Implementation  OK? 

•  .  Evaluate  response 

-  repeat? 

-  accept? 

-  raise  warning  level? 

-  correlate  with  contact 
behaviour? 

-  change  own  tactics? 
-pass  on  to  TG,  etc. 


Communication 

sources 


TG 

ASWC,  SAC,  ORS 
CCS,  State  boards 
Threat  board 
SAC  -  helo  comms 
CO 
OOW 


Workload 

issues 


Could  take  30  mins. 
Depends  on: 

-  team  experience  (if  junior 
so  need  to  watch) 

Traffic  vol. 

Multi-tasking 


Measure 

%  =  complete  +  accurate 


Method 


SA: 

•  Freeze  probes 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 
-real  time  or  video. 

DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 

-  real  time  or  video. 

•  Analysis  of  msg  log 


Comments: 

OR  will  start  audio  recording?  Warnings  may  be  delegated  to  a  warfare  director,  who  may  be  inexperienced.  TG  may  direct  which  warnings  and  when. 
•Cognition: 

Attention:  rapid  switches.  Minimal  processing  time  at  each  switch. 

Memory;  Several  high  memory  load  issues  plus  status  change  detection  as  switch  from  task  to  task. 

•Team  management: 

Smooth  co-ordination  required  to  time  warnings,  evaluate,  and  check  risk  as  contact  closes. 

Review  ROE  for  appropriate  warning. 

•Challenges: 

ORO  may  need  cheat  sheet  if  weak  in  any  warfare  area,  especially  if  warfare  director  is  also  inexperienced. 

May  miscalculate  capability  of  unit  giving  warning  (e.g.  helo) 

May  select  wrong  wng. 
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lEmangiMQB 


-  Ensure  contact  classified  correctl' 


Criterion:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  access  all  info  available  on  contact. 

•  check  contact  info  maximized  (quality  and  quantity). 

•  conduct  own  classification  and/or. 

•  validate  WDs  classification  (check  alternatives,  probe  potential  weaknesses,  recognize  inconsistencies  or  incompleteness 
among  data  or  with  conclusions.) 


Decisions  Communication  Workload  Measure  Method 

_ required _  sources  issues  %  =  complete  +  accurate 

■  Threat  profiles  •  Match  threat  type?  CCS,  GOP,  JMCS  Comfortable  but  tense  SA:  SA: 

1  Classification  criteria  -  confidence  level?  Threat  board  Frequent  switching  among  •  %  classification  criteria  •  Freeze  probes 

All  classification  info  "  oth^  explanations?  WDs  &  OR  tm  members  different  “pictures”  global  .  %  contact  info  .  Embedded  probe 

-confirming  ,  picture,  threat  picture,  and  .  %  threat  info  -  incorrect  data 

-discontinuing  error,  interpretation  error?  Other  TG  assets  vanous  resource  picture.  0.  -wrnnn  ria^ifir: 


Awareness 

needs 


•  Threat  profiles 

•  Classification  criteria 
All  classification  info 

-  confirming 

-  discontinuing 

-  correlation 

•  Potential  error  sources- 

-  sensor  limits 

-  team  experience 

-  ambiguos  data 

-  other  options 

•  All  contact  info 
-IFF 

-  Visual  features 

-  recent  behaviour 
-sensor  data  quality 

Comments: 


•  Match  threat  type?  CCS,  GOP,  JMCS 

-  confidence  level?  Threat  board 

-  other  explanations?  WDs  &  OR  tm  mer 
(friendly  or  neutral,  sensor  ORS,  OOW 
error,  interpretation  error?  Other  TG  assets 

-  verification  needs? 

•  Intended  en  tactics? 

-en  intent? 

-  en  maneuver  plan? 

-  en  schedule  plan? 

•  Classification  correct? 

-  Need  more  data? 


Measure 

%  =  complete  +  accurate 


SA: 

•  %  classification  criteria 

•  %  contact  info 

•  %  threat  info 

•  %  alternatives 

•  %  error  sources 
CM: 

•  comms  @  classification 
-# 

-  duration 

-  content  (?) 

DM: 

•  RT  to  classify 

•  %  classification  errors 


SA: 

•  Freeze  probes 

•  Embedded  probes 
-incorrect  data 

-  wrong  classification 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 

-  real  time  or  video. 

•  Timeline  analysis 
DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 
-real  time  or  video. 


In  practice  ORO  tends  to  rely  on  positive  confirming  correlation  evidence  rather  than  look  for  negative  discontinuing  evidence.  However,  this  can  lead  to  errors.  To 
evaluate  this  may  need  to  set  up  evaluation  situation  in  which  negative  discontinuing  evidence  is  required  to  classify  contact  unambiguously. 

OROs  may  guess  about  en  tactics,  infer  from  en  doctrine,  or  past  patterns. 

Tendency  for  ORO  to  lose  awareness  of  global  picture  (and  possibly  emerging  threats)  while  focussed  on  local  threat  picture. 

OROs  may  fail  to  integrate  current  threat  into  global  picture  and  hence  miss  patterns  such  as  decoy. 

OR  plotting,  reporting,  transcription  errors  common. 
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FOREGROUND  GOALS 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Monitor  recon  track 

-  Confirm  radar  image  OK. 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  + accurate 

Method 

•  Location  and  time  of 
firing 

•  Recon  track  co,  spd,  tk. 

•  Radar  image 

•  Helo  location 

•  Threat  profiles 

•  Confirm  threat  profile 

•  Tgt  of  missile  in  TG? 

•  Helo  safe? 

•  Reckon  track  OK? 

•  Radar  image  OK? 

CCS,  swc, 

SAC,  ORS 

May  “switch  off"  ASWC  to 
reduce  overload. 

SA: 

•  %  Recon  track  info 

CM: 

•  comms  @  track 
comms  @  radar  image 
-# 

-  duration 

-  content  x  class 

DM: 

%  related  decisions 

SA: 

•  Embedded  probe(s) 

•  Freeze  probe(s) 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Post  hoc  freeze  probe 

•  SME  review 

-  real  time  or  video. 

Comments: 
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11-  PRO  goal:  Ensure  detailed  and  viable  tactical  plan  prepared. 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Correctly  analyze  critical  features  of  the  WD’s  tactical  plan  and  identify  errors  or  omissions;  or 

•  Produce  plan  of  own  if  WD  already  fully  engaged. 


Notes: 

This  can  be  viewed  as  a  special  case  of  the  PRO  background  task  of  acquiring  and  maintaining  situation  awareness. 


Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  All  contact  info 

-  CPA,  schedule,  AOP 

•  All  threat  info 

-  capability 
-wpn  ranges 

-  apparent  intent 

-  possible  tactics 

•  Mission  information 
-TG/CO  intent/tasking 
-ROE. 

-  COA  &  pre-plans 

-  procedures 

-  competing  req'ments 

-  impending  req'ments 

•  Own  resources 
-availability 

-  capability 
(e.g.  wpn  ranges) 

•  Timeline 

•  Global  picture 

-  other  contacts 

-  territorial  limits 

-  waterspace  mgt 

-  nav  hazards 
-collateral  damage? 

•  Main  issues  to  check? 
-tasking from  TG? 

•  COA? 

-fits  pre-plan? 

-  alternatives? 

•  1st  re-actions  OK? 

-  maneuver? 

-change  readiness? 

•  Ship  /  OR  ready? 

-  tasks  TBD? 

-  OR  tms  OK? 

•  Free  to  act? 

-  obtain  tasking 

-  ask  for  new  ROE? 

•  Deadlines(s)? 

-for  plan  elements 

CCS,  State  boards, 

GOP,  JMCS 

ORS,  OOW 

WD 

CO,  TG. 

SA: 

•  %  contact  info 

•  %  threat  info 

•  %  mission  info 

•  %  timeline 

•  %  pre-plans 

•  %  Global  picture 

CM: 

•  comms  @  prepare 
-# 

-  duration 

-  content  (?) 

•  Comms  flow 

-  timeliness 

-  queries 

-  tracking  outcome 

DM: 

•  %  ready  to  check 

•  %  plan  options  selected 

•  Decisions  on  critical  path 

SA: 

•  Freeze  probes 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 

-  real  time  or  video. 

comments:  - - - 1 - - — - — 

•Cognition: 

ORO  tries  to  visualize  the  plan(s)  and  options. 

ORO  now  likely  to  have  dropped  background  tasks.  Tend  to  focus  on  this  single  event  and  neglect  big  picture.  May  miss  other  surface  or  air  threats 

May  forget  background  information  such  as  recent  int  messages  or  equipment  malfunction. 

•Team  mgt.: 

Additional  resources  may  be  available  depending  on  state  of  readiness  e.g.  at  Action  Stations. 

•  Potential  challenges: 

iS^?TSrbft^ee!!1aUdJ0  “S  0rder/T  TG  and  txt  messa9e  back  up  that  follows  later.  De-coding  /  transcription  errors  occur  @  5%.  If  so,  queries 
needed  on  voice  to  TG  to  double  check.  (2-3  minutes  between  first  audio  tasking  command  and  text  tasking  message  from  TG.) 
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12  -  ORO  goal(s):  Check  warfare  directors  (WD)  plan. 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Comprehend  WD  plan 

•  Evaluate  WD  plan  in  terms  of  all  strategic  and  tactical  factors 

•  Interact  with  WD  about  plan  (query,  re-direct). 


Awareness 

needs 


•  Warfare  director's: 

-  Global  picture 

-  Mission  &  ROE 
-Threat  picture 

-  Contact  picture 

-  Immediate  goals 
-Timelines 

-  Resource  picture 

-  COA  considered 

-  COA  chosen 

-  Computations 

-  Preparatory  tasks 

-  Risk  assessment 

•  Global  picture: 

-  all  pertinent  factors 

•  Local  picture: 

-  all  pertinent  factors 


Decisions 

required 


•  WD? 

-  cover  all  factors? 

-  info  correct? 

-  info  complete? 

-  considered  all  COA? 

-  calculations  OK? 

-  timeline  OK? 

-  risks  OK? 


Communication 

sources 


Workload 

issues 


Many  others  want  OROs 
attention. 


Measure 

%  =  complete  + accurate 


Method 


SA: 

•  Freeze  probe  after  WD 
briefing. 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 
-real  time  or  video. 

DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 

-  real  time  or  video. 


Comments: 

•ORO  and  Warfare  Director  need  to  see  same  picture  to  interact  effectively.  Need  common  picture  with  no  ambiguity  and  minimal  delays  or  errors  in  understanding. 
•Face  to  face  contact  important  for  body  language.  At  present,  ORO  may  leave  own  CCS  to  confer  at  WD's  CCS  or  GOP. 

If  not  face  to  face,  need  “pointers"  on  each  screen  to  complement  voice  discussions  i.e.  so  ORO  can  point  on  WD  screen  &  vice-versa. 

Preference  expressed  for  round  table  configuration. 

•Style  and  level  of  interaction  varies  with  experience  as  a  team  e.g.  .trust  in  others  judgement. 

•Interaction  between  ORO  and  WD  prone  to  interruption  from  various  voice  sources.  Interaction  and  audio  communicationss  now  more  error  prone. 

•OR  tm  use  hand  signals  to  conduct  internal  communication  traffic  (eg  point  to  headphone  to  indicate  want  to  speak  or  other  to  listen  in.) 

•Experience  needed  to  multi-task  speaking  and  listening  (to  different  nets,  or  net  +  voice)  without  cross  talk  or  omission  errors. 

•Interaction  will  vary  with  different  ORO  and  WD  backgrounds.  Former  SWC  will  have  different  mental  model  and  ask  different  questions  from  former  ASWC 
depending  on  warfare  type  involved. 

•ORO  will  sometimes  miss  internal  OR  voice  communication  because  on  different  net  or  attention  to  CCS  or  text. 

•Mav  turn  down  volume  on  radio  nets  to  listen  to  voices  in  OR  and  forget  to  turn  up  volume  later. 
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Make  CO  aware  of  all  relevant  components  of  the  plan.  Get  feedback  from  CO. 


Criterion:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Brief  CO  on  all  critical  plan  details  clearly  in  shortest  possible  time,  with  as  few  requests  for  clarification  as  possible. 

•  Receive  feedback  /  direction  from  CO  quickly  and  clearly. 

Notes:  ......  . . 

Commonly  will  brief  CO  and  OOW  at  same  time.  CO  and  ORO  will  be  in  OR  together  around  one  CCS  with  OOW  on  the  bridge, 
observing  on  a  slave  CCS  terminal.  Need  common  visual  picture  to  get  information  across  as  quickly  and  effectively  as  possible. 


Awareness 
_ needs 

•  Info  required  by 
CO/OOW 

-TG  tasking  +  goals 

-  Key  factors 

-  Major  options 

-  Outline  plan 

-  Timeline 

-  Risks 

•  Likely  queries 


Decisions 
_ required 

•  What  info  to  choose. 

•  Time  to  brief? 

-when,  how  long? 

•  CO,  OOW  understood? 


Communication 
_ sources 

•  CO,  OOW. 

•  Need  common  picture 
Usually  use  CCS  or  GOP 


Workload 
_ issues 

■  Very  limited  time? 

’  Interruptions  likely 


Measure 

%  =  complete  +  accurate 
"  SA:  (x  ORO) 

•  %-  key  briefing  info 

•  %  -  CO  queries 
SA:  (CO,  OOW) 

•  %-  key  briefing  info 
CM: 

•  Comms  @  brief 

-  #  queries 

-  duration  of  brief 

-  content/type/errors 
DM: 

•  %  key  info  chosen 


Comments: 

CO  may  not  be  in  OR  to  view  screen  so  ORO  may  have  to  paint  visual  picture  verbally.  Such  verbal  info  prone  to  error. 
Need  to  confirm  that  OOW  has  understood  all  points.  Will  select  information  to  match  CO  and  OOW  style,  experience,  etc. 

•  Research  question: 

how  best  to  rapidly  communicate  critical  plan  information  among  team. 


Method 


SA: 

•  Freeze  probe 

-  pre-brief 

-  post-brief  (to  CO) 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  SME  (CO)  review 

-  post  hoc,  real  time,  or 
video. 
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14  -  ORO  Goal  (s)  Ensure  all  OR  team  aware  of  plan.  Emphasize  key  points. 


Criterion:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  make  OR  team  members  aware  of  critical  plan  details  quickly  and  clearly 

(i.e.  in  shortest  possible  time,  with  as  few  requests  for  clarification  as  possible.) 

•  confirm  that  OR  team  comprehends. 


Notes: 

Currently  everyone 
briefing  is  provided, 
be  a  time  of  stress. 

Awareness 

needs 


All  attending? 

Info  required  by  OR 

•  TG  tasking 

•  Ship's  goal 

-  Outline  plan 


■  Key  OR  task  points 
Likely  queries 
Comprehension 


•  OR  tm: 

Face  to  face  -  voice. 
All  attend  with  head 
phones  off. 


Method 


in  the  OR  who  is  not  on  a  critical  task  has  to  remove  headset  from  at  least  one  ear  and  attend  to  the  ORO  as 
Some  OR  members  may  continue  to  watch  screen  or  monitor  a  communication  network  with  one  ear.  Likely  to 
Used  to  focus  the  OR,  provide  specific  information,  and  to  confirm  ‘command  presence’.  _ 

Decisions  Communication  Workload  Measure  Method 

required  sources _ issues _ %  =  complete + accurate _ 


•  What  info  to  choose? 

•  Time  to  brief? 

-when,  how  long? 

•  OR  tm(s)  understand? 


Workload 
_ issues 


'  Pick  'quiet'  moment, 
i  Limited  time 
'  Takes  30-40  seconds 


Measure 

=  complete  +  accurate 


•  Freeze  probe: 

-  pre-brief  (ORO) 

-  post-brief  (OR  tm) 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  SME  (OR  tm?)  review 

-  post  hoc,  real  time,  or 
video. 


Comments: 

Typically  only  required  for  refocusing  team  and  setting  priorities  in  multi-threat  contexts. 

Standard  procedure  for  briefing  learned  during  training. 

Graphics  quickest 

Verbal  briefings  prone  to  error. 

Interruptions  likely.  OR  team  members  may  be  distracted  with  ongoing  tasks. 

•Evaluation 

Need  to  rate  “style''  as  well  as  content.  To  establish  how  well  content  transformed  could  use  'back  briefing’  approach. 
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15  -  PRO  GoaKsl:  Supervise  execution  of  plan  bv  OR  team. 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Know  the  plan  and  the  status  of  its  implementation  (completeness,  quality,  appropriateness). 

•  Recognize  errors  omeed  for  change  and  correct  promptly. 

•  interact  with  OR  tm  members  as  appropriate  . 

•  Recognize  overload  within  the  OR  promptly  and  re-direct  OR  team  efforts. 

•  Be  able  to  provide  back  up  personally  when  required  (e.g.  second  threat  emerges.). 


Notes:  While  monitoring  plan  execution,  the  ORO  needs  to  maintain  global  awareness  in  order  to  be  able  to  redirect  the  plan  if  the 
big  picture  dictates,  and  to  be  ready  to  react  to  other  threats  or  tasking.  See  ‘Background  tasks’  for  more  information.  ORO  tries 
to  anticipate  and  recognize  problems  and  intervene  or  assist  but  avoid  interference. 


Awareness  Decisions  Communication  Workload  Measure  Method 

_ needs _ required _  sources  issues  %  =  complete + accurate 

•  Plan  •  Planned  tasks  CCS,  State  boards,  Multi-tasking  SA:  (x  ORO)  SA: 

- goal(s)  -on  schedule?  GOP,  JMCS  High  stress  •  %  Plan  •  Freeze  probes 

-tasks  to  be  done  -qualilyOK?  WD  High  workload  -goal(s)  .  SME  review 

-outcome  OK?  ORS  OOW.  -tasks  to  be  done  -reel  time  or, idea 

-  scneauie  .  •  Intervene?  SIGS,  CSE  _  schedule  pm- 

-task  completion  -change  plan?  CO,TG.  -  task  completion  c'.c  . 

-task  quality  -  add  resources?  -emerging  problems  •  SME  review 

-  OR  wkload  problems  -  give  guidance?  •%  global  picture  -real  time  or  video. 

•  Global  picture  •  Report  required?  -  new  int,  contacts  L.'  , 

-  new  int,  contacts  -from  OR  tm  -  effect  on  execution  •  Embedded  probes 

-effect  on  plan  -  to  TG,  CO,  etc.  -  emerging  risks  -realtime 

-  emerging  risks  •  Global  picture  OK?  -  emerging  threats  * review 

-emerging  threats  CM;  -  real  time  or  video. 

•  Comms  @ 

implementation  *  WL  sca,e 

-  duration  "  S#  after  action. 

.#  -SME 

-  content/type(?)  •  I '  (rea'  *'me  or  video) 
DM: 

•  %  problems  identified. 

- - - - - »  %  solutions  OK?. 

Comments:  T  .  ~~  '  '  — 

Feedback  from  “feel"  of  ship  (heel,  speed,  etc)  very  important  now.  ORO  needs  “sense"  of  movement  in  space.  May  need  reminders  for  course  changes  due  ( e.g. 
route,  search,  or  zig  zag).  Lost  contact  can  occur,  and  need  search  plan  templates  when  this  happens.  May  need  to  revise  search  plans  for  local  conditions  -  e.q 
territorial,  limits,  civil  traffic,  collateral  damage  potential. 

Tries  to  visualize  underwater  or  external  picture.  May  query  the  OOW,  or  use  STIR  cameras  if  available.  External  360  degree  picture  would  help. 

ORO  needs  to  know  when  change  occurs  in  plan  and  update  system  as  these  occur,  e.g.  schedule  changes?  Will  these  be  automatic? 

Current  CCS  plot  frequently  incorrect:  always  look  for  anomalies  between  sensors  as  a  cue.  Symbol  transcription  errors  common. 

Team  can  omit  pre-plan  step  in  excitement.  Smart  prompts  on  possible  actions  would  help  memory. 

Tunnel  vision  more  associated  with  ASW  during  this  time  as  more  ambiguities. 


Decisions 

required 

•  Planned  tasks 

-  on  schedule? 

-  qualily  OK? 

-  outcome  OK? 

•  Intervene? 
-change  plan? 

-  add  resources? 

-  give  guidance? 

•  Report  required? 

-  from  OR  tm 
-toTG,  CO,  etc. 

•  Global  picture  OK? 


Communication 
_ sources 

CCS,  State  boards, 
GOP,  JMCS 
WD 

ORS,  OOW, 

SIGS,  CSE 
CO,  TG. 


Workload 


Multi-tasking 
High  stress 
High  workload 


Method 

SA: 

•  Freeze  probes 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 

-  real  time  or  video. 
DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 

-  real  time  or  video. 
WL: 

•  WL  scale 

-  S#  after  action. 
-SME 

(real  time  or  video) 
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16  -  ORO  Goalfs):  Ensure  standard  procedures  completed  correctly  (ZIPPO,  AIR  RED,  Action  Stations,  Greyhound,  etc) 
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Annex  C: 
FOREGROUND  GOALS 


17  -  ORO  Goal  (s)  Stav  on  top  of  global  picture. 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

•  Switch  readily  between  monitoring  threat  response  and  the  global  picture. 

•  Cross  reference  changes  or  occurrences  in  either. 

•  Deal  with  other  occurrences  (e.g.  another  threat,  equipment  failure,  unexpected  navigation  hazard  )without  losing  either  the 
threat  or  global  picture. 

Notes:  This  can  be  regarded  as  part  of  the  background  task  of  maintaining  background  and  foreground  awareness. 


Awareness 
_ needs 

•  Plan 

-  elements 

-  schedule 

•  Ship  movement: 

-co,  spd  (repeaters), 

-  OOW  response 

-  response  (feel) 

•  Environmental  effects: 

-  underwater, 

-  surface 

-  coastal  terrain 

-  relative  wind 

•  Target  details 

-  current  position 

-  current  co,  spd,  etc 

-  projected  details 

-  any  changes 

•  Relative  locations: 
-target 

-contacts  in  area,  TG, 

-  nav  hazards 

-  limits  (territorial,  etc) 

•  Emerging  threats 


Decisions 

required 


•  Desired  co  &  spd. 

-  Helm  orders  (?) 

-  Ship  safe? 

-Aim  being  achieved? 


Communication 


Workload 


CCS,  Frequent  switching 

OR  repeaters  -  co,  spd  rel  between  BP  and  threat 


wind 

OOW 

OR  tm  Idrs  (for  info) 


picture(s).  WL= 
comfortable  but  tense. 
May  continue  for  hours. 
May  split  con  with  CO. 
Others  in  OR  alert  ORO  if 
something  else. 


Measure 

%  =  complete  +  accurate 


SA: 

•  %  plan  elements 

•  %  timeline 

•  %  target  details 

•  %  en  intent 

•  %  ship  response 

•  %  environment 

•  %  contact  locations 

•  %  procedures 

•  %  global  picture 
CM: 

•  comms  @  maneuver: 
-# 

-  duration 

-  content  (?) 

DM: 

•  RT  significant  changes 

•  %  significant  changes 

•  %  reactions  correct 
WL: 

•  Red  line: 

- #  exceed 

-  duration  exceed 
-effect  on  other  tasks 


Method 


SA: 

•  Freeze  probes 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 

-  real  time  or  video. 
|DM: 

•  Embedded  probes 

-  real  time 

•  SME  review 

-  real  time  or  video. 
WL: 

•  WL  scale 

-  S#  after  action. 
-SME 

(real  time  or  video) 


Comments: 
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18  -  ORO  Goalls):  Back  up  Warfare  Director  (WD)  as  required 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Monitor  what  the  WD  is  doing. 

-  Check  WD’s  actions. 

-  Intervene  as  required. 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Contact  details 

•  Procedural  steps 

•  WDs  progress 

•  VETO  set  up 

•  Chaff  release  plan 

•  TG  plan  and  capabilities 
(eg  ATH  =  long  range 
missile  defence.) 

•  WD  workload 

•  WD  progress  OK? 

•  Take  over  from  WD? 

•  Ask  TG  to  take  if  in 
range? 

•  Enable  VETO 

WD 

CCS 

TG 

%  Detection  and  correction 
of  WD  errors. 

Video  analysis  of  scenario. 

Comments: 

•Cognition: 

Currently,  too  much  attention  on  mechanics  of  display  management.  Distracts  from  main  awareness 

•Potential  Problems:  ,  ..  ,  .  ,  . 

Need  to  range  down  CCS  to  cover  missile  approach  on  screen  This  does  not  always  happen  smoothly.  Can  become  disoriented  as  switch  takes  place. 

Error  diagnosis  for  SWC:  freezes,  misses  key  diagnostic  point,  wrong  QAB  activated  (eg  single  vs  salvo),  keying  error. 

•Relevant  Experience: 

Novices  forget  VETO.  Know  TG  capabilities  e.g.  ATH  has  longer  range  for  radar  and  missiles. 
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19  -  ORO  Goal(s):  Maneuver  ship  as  required. 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Determine  desired  course  and  speed 

-  Check  suitability  (hazards,  objectives,  etc) 

-  Direct  desired  course  and  speed 


Awareness 

needs 

Decisions 

required 

Communication 

sources 

•  Global  picture 

•  Co  &  spd  to  take 

•  OOW 

-  current  pos'n  /  track 

-now 

•  CCS 

-  future  pos’n /track 

-future 

•  Indicators 

•  Threat  picture 

-Turn  and  course 

-current  pos'n /track 

-speed 

-  future  pos’n  /  track 
•  Feedback  @: 

-  Orders  understood 
-Course  changes 

-  Speed  changes 

-  ship  movement 

Workload 

issues 


WL  =  High  when 
maneuvering.  Can,t 
maintain  level  longer  than 
an  hour.  With  6  other  ships 
at  high  speed  becomes 
major  priority  -  esp  at  night 
and  bad  sea  state.  White 
shipping  #1  priority. 


Measure 

%  =  complete  + accurate 


%  Maneuver  goals 
%  current  track 
%  future  track 


Method 


im 


•  Embedded  probe(s) 

•  Freeze  probe(s) 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 

-  real  time  or  video. 
DM: 

•  Embedded  probe. 
-RT:  maneuver  errors 

•  SME  review 

-  real  time  or  video. 


Comments: 

•Evaluation  Methods: 

Need  to  Insert  errors  in  co,  speed  or  schedule  into  evaluation  scenario  to  evaluate  response  to  such  problems,  and  measure 
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20  -  PRO  Goal(s'):  Check  for  other  threats 

See  “Background”  and  “Generic  Threat”  tasks 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 
-  Detect  any  other  risk  or  threat  to  the  ship  or  to  the  mission 


Awareness 

needs 


Decisions 

required 


g  picture? 
In? 


Communication 

sources 


TG,  external  AAW, 
CCS  screen. 


Workload 

issues 


Likely  to  be  very  high  if 
threat  under  way.  Stress 
levels  high  too. 


Measure 

%  =  complete  +  accurate 


|See  previous  serials 


Method 


See  previous  serials 


Comments: 
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21  -  PRO  Goal(s):  Respond  to  second  threat 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 
-  Respond  in  the  same  way  as  for  threat  #1.  See  items  1  -  20  above. 


Awareness 

needs 

Decisions 

required 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

IComments: 


•Cognition: 

Time  sharing  between  2  warfare  areas.  Likely  to  lose  focus.  Frequent  switching. 

•Potential  Problems: 

ORO  multi  tasking  under  stress.  Must  be  able  to  focus  on  new  threat,  monitor  existing  threat,  and  maintain  awareness  of  global  picture. 
•Team  issues: 

May  handover  one  or  more  tasks  to  others  in  OR  for  example  CO,  off  watch  ORO,  Combat  Officer,  ASWC  if  capable  and  not  already  busy. 
Problem  that  not  enough  consoles. 
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Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Determine  that  enemy  target  has  been  destroyed 

-  Confirm  that  no  further  threat  exists. 


22  -  ORO  Goalfs):  Confirm  enemy  target  destroyed 


iComments: 


Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Engagement  details 

•  All  sources  of  data. 

-  EW,  radar,  visual. 

•  Threat  gone? 

•  Correlate  EW,  radar, 
visual  info? 

•  Fire  own  solution? 

•  Further  action 

WD,  CCS,  OOW,  TG. 

Very  little  time. 

SA: 

•  %  engagement  details 

•  Correlation  pattern 

CM: 

•  Comms  @  engagement 
-  # 

-  duration 

-  content  x  class 

DM: 

%  Assessment 

SA: 

•  Embedded  probe(s) 

•  Freeze  probe(s) 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Post  hoc  freeze  probe 
SME  review 

-  real  time  or  video. 

May  need  to  continue  to  monitor  capability  of  partially  neutralized  threat. 
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23-  ORO  Goalfs):  Assess  damage  to  own  ship  from  missile  (etc)  hit 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Determine  pertinent  damage. 

-  Assess  functional  impact  on  OR  capabilities 

-  Assess  repair  schedule  and  completeness 

-  Determine  work  required  /  possible 

-  Determine  impact  on  current  TG  tasking 

-  Make  recommendations 


Awareness 

needs 


•  Lost  capability 

-  sensors 

-  weapons 

-  personnel 

-  communications 

•  Consequence  for  task(s) 

•  Repair  schedule 


Communication 


Decisions 

required 


>  Work  around  ship  or  TG?  WD,  OOW,  ORS,  CSE 


Workload 


sources 


Measure 

%  =  complete  +  accurate 


Method 


IVery  little  time. 


-  weapons 

-  personnel 

-  communications 

•  Continue  tasking? 

•  Recommendations? 
-CO 

-TG. 


CCS,  Comms  panel 
CO,  TG. 


im 


•  %  capability  lost 

•  Comms  @  damage 
-  # 

-  duration 

-  content  x  class 
DM: 

%  effect  on  tasks 
%  recommendations 


im 


•  Embedded  probe(s) 

•  Freeze  probe(s) 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 

-  real  time  or  video. 
DM: 

•  Post  hoc  freeze  probe 
SME  review 

-real  time  or  video. 


Comments: 
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o/t  _  non  r-noir<n  _  Retrain  awareness  of  the  global  picture  following  period_of  concentration  at  threat  level. 

ORO  Criteria:  An  O 

•  Recognizes  whe 

•  Regain  global  Si 

•  Recognize  chanj 

Notes:  The  need  to  n 

RO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 
a  to  switch  to  global  picture, 
tuation  Awareness. 

res  in  Situation  since  last  attended,  and  their  significance. 

;-acquire  awareness  in  one  domain  after  a  period  of  intense  concentration  in  another  frequently  occurs  and  is  not 
For  the  ORO  this  may  be  a  switch  from  threat  area  to  threat  area,  after  ‘zooming’  out  or  in,  or  when  changing 
to  the  threat  picture,  or  some  other  aspect  such  as  equipment  capability,  an  incoming  flash  message,  a  damage 
re  else  in  the  ship  that  imposes  limitations  on  plan  implementation. 

peculiar  to  the  ORO. 
focus  from  the  global 
report  from  somewhe 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  When  to  switch  to 
global  picture. 

•  Changes  since  last 
inspected. 

•  Where  to  attend. 

•  Significance  of  changes 

•  CCS,  GOP,  JMCS, 

SWC,  ORS,  ASWC 

TG 

As  required 

SA: 

•  spatial,  content  and 
temporal  information 
%TG, 

•  %  all  warfare  areas, 

«  %  relationship  of  new  to 
existing 

•  %comm  traffic. 

•  RT  to  recover 
information. 

M: 

•  %sources  consulted 

•  %data  selected 

SA: 

•  Probe-active/passive 

•  SME  review  of  overt  task 
behaviour. 

Comments: 

global  picture  (or  vice-versa).  Need  to  comprehend  pertinent  changes  since  last  'view'  as  quickly  as  possible. 

•Cognition: 

When  switch  attention  to 
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25-  ORO  Goal(s):  Re-focus  OR  team.  Calm  everyone  down.  Re  establish  command  presence. 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Pick  key  items  to  brief  OR  team. 

-  Re-focus  ongoing  surveillance 

-  Ensure  that  OR  back  on  task  (see  “Manage  OR”) 


Awareness 

needs 


Way  forward 
Tasks  in  hand 


Decisions 

required 


•  What  to  brief 

-  new  ROE 

-  active  pre-plans 

-  political  changes. 

•  Lessons  learned 


Communication 

sources 


•  OR 

•  Back  row  team 

•  OOW 


Workload  Measure 

issues  %  =  complete  +  accurate 


SA: 

•  %  Briefing  needs 
CM: 

•  Comms  @  to  re-focus 
-  # 

-  duration 

-  content  x  class 
DM: 

%  briefing  content 
Rating  briefing  presence? 
RT  -  start  briefing 
RT  -  OR  back  on  task 


Comments: 

Could  change  several  ROE  because  have  now  been  fired  on. 


Method 


jSA: 

•  Embedded  probe(s) 

•  Freeze  probe(s) 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 
[DM: 

•  Post  hoc  freeze  probe 
|SME  review 

-real  time  or  video. 
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26  -  ORO  Goal(s):  Stand  down  own  weapons 


Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  confirm  that  there  are  no  other  threats 

-  stand  down  own  missiles  or  destroy  in  air. 


Awareness 

needs 

Decisions 

required 

Communication 

sources 

•  Other  threats 

•  Confirm  no  threats? 

WD,  ORS,  OOW,  TG 

•  Own  weapons  fired 

•  Switch  to  other  threats? 

CO 

-  details 

•  Own  weapons 

-  destroy  missiles  in  air? 

-  stand  down  wpns? 

•  Stand  down? 

-which? 

-  when? 

Workload 

issues 


Measure 

%  =  complete  +  accurate 


Method 


SA: 

•  Embedded  probe(s) 

•  Freeze  probe(s) 

•  SME  review 

-  real  time  or  video. 
CM: 

•  SME  review 

-  real  time  or  video. 
DM: 

•  Post  hoc  freeze  probe 

•  SME  review 

-  real  time  or  video. 


Comments: 

May  need  to  monitor  partially  neutralized  threat. 
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28  -  ORO  Goalfs):  Report  to/from  TG 

Criteria:  An  ORO  must  be  able  to,  in  a  timely,  complete  and  accurate  way: 

-  Determine  and  schedule  all  reports  required  (internal  and  external). 

-  Collate  and  check  information  required  by  reports. 

-  Assemble  and  transmit  reports  according  to  schedule. 

Awareness 

needs 

Decisions 

required 

Communication 

sources 

Workload 

issues 

Measure 

%  =  complete  +  accurate 

Method 

•  Reports  required 

•  Report  schedule 

•  Report  content. 

•  Report  list  complete? 

•  Report  schedule  OK? 

•  Report  OK  for  TX? 

-  content 
-format. 

TG 

WD 

ORS,  Comms  staff/ 

SA: 

•  %  report  requirements 
CWI: 

•  Comms  @  reports 
-  # 

-  duration 

-  content  x  class 

DM: 

%  reports  missed  /  late 
%  content  incorrect 

RT  -  report  preparation 

SA: 

•  Embedded  probe(s) 

•  Freeze  probe(s) 

•  SME  review 

-  real  time  or  video. 

CM: 

•  SME  review 

-  real  time  or  video. 

DM: 

•  Post  hoc  freeze  probe 
SME  review 

-  real  time  or  video. 

Comments: 

Currently  have  to  transcrib 

;  most  information  by  hand  from  various  screens  around  OR. 
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Annex  D: 

SCENARIO  EVENT  SEQUENCES 


Scenario  event  sequences 

Introduction 

In  this  Annex,  activities  for  two  scenario  events  requiring  active  threat  engagement  (i.e.  weapons  are  fired)  are 
tabulated  and  cross  referenced  to  the  data  tables  in  Annex  A,  B  and  C.  Note  that  the  description  of  the  goals  in 
these  tables  sometimes  differ  slightly  and,  in  many  cases,  cross-reference  to  more  that  one  item  in  the  data 
tables.  This  is  because  the  goals  related  to  scenario  event  items  may  have  been  expressed  differently  during 
interviews.  Nonetheless  the  underlying  intent  appeared  to  be  the  same  as  that  expressed  in  the  generic  goals  in 
the  data  tables.  When  setting  up  measurement  profiles  for  a  given  scenario,  generic  descriptions  and 
measurements  taken  from  the  data  tables  will  need  to  be  customized  and  brought  into  focus  for  the  specific 
circumstances  set  up  in  the  scenario  (time,  place,  environment,  threat  specifics,  etc). 

Table  D1  collates  all  ORO  goals  from  Annex  A,  B  and  C.  Table  D2  cross-references  goals  to  event  activities 
for  Event  #  6  -engagement  with  a  shore  missile  batteiy.  Table  D3  cross-references  goals  to  event  activities  for 
Event  #7  -  engagement  with  an  enemy  surface  action  group. 
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Annex  D: 

SCENARIO  EVENT  SEQUENCES 


Generic  ORO  goals  /  tasks 

(Not  all  needed,  &  sequence  only  approximate.  Row  entries  are  not  related  across  columns.) 


Annex  A.  Coming  On  watch 


1 .  Visualize  external  environment  for  the  start  of 
the  watch. 


2.  Update  own  knowledge  for  coming  watch 
from  text  msq  information. 


3.  Establish  schedule  of  anticipated  events  for 
watch. 


4.  Update  understanding  of  threats. 


5.  Establish  current  activity  levels  &  work  focus 
among  OR  team(s) 


6.  Prepare  check-list  to  monitor  progress  on 
required  action  items  during  watch 


7.  Review  OR  system  performance  (sensors, 
etc)  &  rectify  or  work  around  as  required 


8.  Comprehend  items  in  CO’s  order  book. 


9.  Comprehend  brief  from  outgoing  ORO 


lO.Optimizeown  CCS . 


1 1 .  Focus  OR  team  on  period  ahead. 
Establish  command  Dresence. 


Annex  B.  Background 


1.  Monitor  &  manage  OR  team(s). 


2.  Manage  information  from  all  sources  coming 
to  ORO  position  (speech,  text,  other). 


3.  Monitor  &  manage  OR  equipment  capability. 


4.  Build,  monitor  &  maintain  own  awareness  of 
pertinent  surface  environment. 


5.  Build,  monitor  &  maintain  own  awareness  of 
pertinent  air  environment. 


6.  Build,  monitor  &  maintain  own  awareness  of 
pertinent  sub-surface  environment. 


7.  Build,  monitor  &  maintain  own  awareness  of 
integrated  picture(s)  comprising  selected 
elements  of  surface,  sub-surface  &  air 
environments  &  other  related  information 
(e.q.  resources). 


8.  Switch  attention  between  different  tasks  & 
acquire  awareness  quicklv  &  effectively. 


9.  Maintain  awareness  of  ORO's  authority  to 
act. 


lO.Co-ord  with  other  TG  members 


1 1  .Check  outgoing  message  traffic. 


Annex  C.  Generic  threat  response 


1.  Integrate  new  contact  information  with  own 
global  picture 


2.  Ensure  OR  prepares  quick  first  response, 
broad  plan  for  worst  case. 


3.  Optimize  work  among  OR  team(s) 


4.  Optimize  information  flow  within  &  outside 

OR. _ _ 

5.  Maximize  number  of  information  sources 
concerning  contact 


6.  Optimize  data  quality  for  each  information 
source 


7.  Establish  communication  with  contact  to 
resolve  identity  &  intentions 


8.  Conduct  warnings  according  to  TG  tasking. 


9.  Ensure  contact  classified  correctly 


lO.Ensure  OR  detects  if  en.  missile  (etc)  fired  & 
establish  firm  track. 


1 1 .  Ensure  detailed  viable  tactical  plan 
prepared. 


12.  Check  warfare  directors  (WD)  plan. 


13.  Make  CO  aware  of  all  relevant  components 
of  plan.  Get  feedback  from  CO. 


14.Ensure  all  OR  team  aware  of  plan. 
Emphasize  key  points. 


15.Supervise  execution  of  plan  by  OR  team. 


16.  Ensure  standard  procedures  completed 
correctly  (Greyhound,  ZIPPO,  Action 
stations,  ASMD,  etc). 


17.  Stay  on  top  of  global  picture  during  threat 
response. 


18.  Back  up  WD  director  as  required. 


19.  Maneuver  ship  as  required. 


20.  Check  for  other  threats 


21 .  Respond  to  second  threat 


22. Confirm  enemy  target  destroyed 


23.  Assess  own  ship  damage  from  missile  hit. 


24.  Regain  awareness  of  global  picture  following 
period  of  concentration  at  threat  level. 


25.  Re-focus  team.  Establish  command 


ifrAJwMiT.Er.'mara; 


27.  Stand  down  shi 


28.Complete  &  send  reports. 


Table  Dl:  Generic  Coming  on  Watch,  Background  and  Threat  related  ORO  goals 
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Annex  D: 

SCENARIO  EVENT  SEQUENCES 


Scenario  Event  Item 

ORO  Goal(s) 

Data  Table  # 

Event  #6:  Shore  missile  battery  engagement 

1.  TORONTO  detects  EW  racket.  Classifies  as 

RICEBOWL  in  search  mode.  EWS  reports  to  SWC. 

C3 

2.  Optimize  OR  info  qathering  @  contact. 

C4,5,6 

3.  Prepare  for  worst  case. 

C2 

4.  Ensure  OR  team  members  remain  focussed  on  own  tasks.* 

C3,C4 

2.  TG/EWC  correlates  with  known  battery  location. 

5.  Cross  check  data  from  TG  with  own  threat  picture 

C5 

3.  RICEBOWL  EW  racket  changes  to  target  mode. 

6.  Ensure  OR  detects  missile  firinq  &  establish  firm  radar  track. 

CIO. 

7.  Monitor  key  issues  &  chanqes  in  qlobal  picture. 

B4-7 

8.  Supervise  execution  of  pre-plan 

Cl  5 

4.  TG/MWC  directs  ZIPP03  and  AIR  THREAT  RED. 

9.  Ensure  procedures  completed  correctly 

Cl  6 

10.  Maneuver  ship  as  required. 

Cl  9 

5.  TORONTO  goes  to  ACTION  STATIONS. 

1 1 .  Ensure  Action  stations  procedures  completed 

C16 

12.  Brief  ship  about  reason  for  going  to  Action  Stations  (etc). 

—  limn . 

6.  ORO  checks  OR  response  to  ZIPPO  requirements. 

ORO  waits  for  CHAFF  report 

13.  Ensure  procedures  completed  correclty 

C16 

14.  Check  SWCs  Fire  control  solution* 

C12 

15.  Check  ASMD  (Anti  Ship  Missile  Defence) 

Cl  6 

16.  Maneuver  ship  for  ASMD. 

Cl  9 

7.  Radar  shows  two  contacts. 

17.  Confirm  contact. 

C9 

8.  SWC  resolves  tracks 

9.  SWC  assess  tracks  as  incoming  missiles 

Recommends  solution  to  ORO 

18.  Check  SWC  resolve  &  fire  control  solution. 

Cl  2 

19.  Brief  CO  and  OOW 

Cl  3 

20.  Brief  OR 

C14,  All 

21.  Check  that  OR  &  OOW  all  OK. 

B1.C14 

22.  Co-ord  with  TG. 

B7,  BIO 

23.  Back  up  SWC  while  implementing  firing  solution. 

Cl  8 

10.  Athabascan  engages  both  missiles 

24.  Track  missiles  fired  from  ATHABASKAN. 

CIO,  C11 

Maintain  readiness  of  own  ship  solution. 

25.  Check  for  second  threat. 

B7,  B8 

11.  Missiles  destroyed 

26.  Confirm  enemy  missiles  destroyed. 

C5,  C6 

27.  Stand  down  own  weaponsmissiles. 

C26 

28.  Recapture  big  picture. 

B7,8 

12.  TG/AAWC  cancels  ZIPPO 

29.  Re-focus  OR  team.  Re  establish  command  presence. 

C14.A11 

13.  Toronto  stands  down  from  Action  Stations. 

30.  Stand  down  ship. 

C27 

31.  Report  to/from  TG 

B11.B28 

Table  D2;  Scenario  Activities  for  Event  #  6,  PRO  Goals,  and  related  data  tables 
(Data  Table  #’s  refer  to  goals  in  Annex  A,  B,  or  C) 
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Annex  D: 

SCENARIO  EVENT  SEQUENCES 


Scenario  Event  Item 

•L 

ORO  Goal(s)  Datat  Table  # 

_m _ i _ _ 

Event  #7:  Surface  Action  Group 

• 

1.  TORONTO  receives  LINK  from  MPA. 

Two  unknown  tracks  @  80  nm  from  ship. 

•  Prioritize  own  efforts  Cl ,  C2,  B4,  B7 

•  Optimize  OR  information  gathering  C4,  C5,  C6 

•  Prepare  broad  plan  for  worst  case  C2 

2.  TG/ASUWC  tasks  MPA  to  further  investigate. 

•  Wait  for  MPA  data  NA 

3.  MPA:  verbal  report  that  tracks  3  nm  apart  at  25  kts  heading  to  TG. 

•  Classify  contact  C9 

•  Brief  OR  tm  A11.A14 

4.  MPA  verbal  report:  EW  racket  for  air  search  radar  from  one  contact. 

•  Re-dassify  contact  C9 

5.  TG/ASUWC  classifies  trks  as  hostile.  MPA  responsible  for  LINK. 

•  Continue  preparation  NA 

6.  TG  comdr  directs  AIR  and  SURFACE  threat  warning  RED. 

•  Start  AIR  and  SURFACE  RED  C17 

•  Ensure  weapons  ready  Cl  5 

7.  ORO  recommends  ACTION  STATIONS. 

ORO  monitors  and  checks  status  as  ship  goes  to  AS. 

•  Implement  ACTION  STATIONS  C17 

8.  TG:  tasks  TORONTO  +VDQ  as  SAG  +  MPA  as  Tgt  Report  Unit  (TRU). 

•  Brief  TRU  Cl  4,  All 

9.  ORO  informs  CO  +  SWC  +  ORS.  TRU  follows  Greyhound  procedures. 
ORO  monitors  G-hound. 

•  Implement  Greyhound  Cl  7 

•  Estimate  times  to  missile  firing  Cl  1 

10.SWC  prepares  plan.  SWC  briefs  ORO  and  CO  then  OR  and  TRU 

•  Check  SWCs  plan  Cl  2 

11. ORO  conducts  own  checks. 

•  Check  safety  issues  Cl  2,  Cl  5 

•  Solution  checks  to  backup  SWC  fire  soln.  Cl  1,  C12 

•  Plan  after  missiles  fired.  C11.C12 

12.  SWC  confirms  ready  to  fire.  CO  gives  permission. 

SWC  fires  missiles  at  en  ships. 

•  Ensure  missiles  launched  Cl  5 

•  Back  up  SWC.  Cl  8 

•  Maneuver  ship  as  required  Cl  9 

•  Track  own  missiles  CIO 

13.  MPA:  SLAMEYE  report  =  2  missiles  from  en  SAG. 

•  Ensure  RECKON  track  in  place.  CIO 

•  Prepare  fire  solution  to  back  up  SWC  C12,  C18 

•  Coord  TRU  B7,  B8,  B10 

14.TG/AAWC  calls  ZIPP04. 

•  Implement  and  check  ZIPP04  C15,  C17 

15.  ORO  monitors  ZIPP04,  readiness  reports  from  ORS,  SWC,  EWS. 
Reports  Z4  complete  to  CO. 

16.Ships  radar  paints  missiles  at  20  nm.  Establish  track  (from  SGI  50). 
OOW  reports  visual  contact  with  missiles  at  10  nm. 

•  Evaluate  track.  CIO,  Cl  1 

17.  SWC  resolves  contact  and  calculates  CPA.  Final  check  with  ORO. 

SWC  asks  permission  to  fire  from  CO  (via  ORO) 

•  Double  check  SWC  solution  Cl  2 

•  Back  up  SWC  firing  actions.  Cl  8 

18.  EWS  detects  missile  lock  on  radar.  ORO  checks  chaff  gone. 

•  Back  up  SWC  Cl  8 

19.  SWC  confirms  permission  to  fire.  Fires  two  Sea  Sparrow  missiles. 

•  Back  up  SWC  C18 

20.  ORO  monitors  engagement. 

One  en.  missile  destroyed.  Second  en.  missile  not  destroyed. 

•  Monitor  incoming  missile.  C4,  C5,  C6 

•  Plan  for  worst  case.  C2,  C11 

21.  MPA  reports  SNOOPTRAY  EW  racket. 

•  Maximize  info  @  contact,  C4,  C5,  C6,  B6. 

B7,B8 

•  Plan  for  worst.  C2 

•  Coord  OR  response.  B1.B10 

22.CIWS  automatically  engages  (5)  1500  yds. 

•  Focus  on  ASW  problem.  C4,  C5,  C6 

23.CIWS  succesful. 

Damage  =  UHF:  405  (AAW  coord).  210  (TG  tac  net). 

213  (MPA  task  gp  report  +  LINK  net). 

Reconfigure  comms. 

•  Assess  damage  C23 

•  Work  around  damage  C23 

•  Assess  ability  to  fulfill  SAG  task.  C23 

•  Maintain  OR  focus  on  ASW  threat  CIO,  B10,  B11 

24.  MPA  reports  DAMCAT  on  own  missile  firing:  both  en  SAG  contacts 
dead  in  water,  smoke  from  one. 

25.  Pass  DAMCAT  to  TG/ASUWC. 

•  Prepare  DAMCAT  B2.B11 

•  Check  DAMCAT  B2.B11 

•  Recommendations  to  TG.  CIO,  B10,  B11 

26.TG/SWC  terminates  SAG  and  Greyhound. 

•  Recapture  big  picture  B7,  B8 

•  Calm  &  re-focus  OR.  Establish  command  presence.  A11.C14 

•  Stand  down  ship  C27,  C26 

27. ORO  prepares  reports.  Voice  and  hard  copy. 

•  Check  reports  complete  and  accurate  B2,  B1 1 

Table  D3:  Scenario  Activities  for  Event  #7,  ORO  Goals,  and  related  data  tables 


(Data  Table  #’s  refer  to  goals  in  Annex  A,  B,  or  C) 
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